llm-ollama 0.1.0

LLM plugin providing access to local Ollama models using HTTP API

llm-ollama

LLM plugin providing access to models running on local Ollama server.

Installation

Install this plugin in the same environment as LLM.

llm install llm-ollama

Usage

First, ensure that your Ollama server is running and that you have pulled some models. You can use ollama list to check what is locally available.

The plugin will query the Ollama server for the list of models. You can use llm ollama list-models to see the list; it should be the same as output by ollama list. All these models will be automatically registered with LLM and made available for prompting and chatting.

Assuming you have llama2:latest available, you can run a prompt using:

llm -m llama2:latest 'How much is 2+2?'

To start an interactive chat session:

llm chat -m llama2:latest

Chatting with llama2:latest
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
>

Model options

All models accept the following options, using -o name value syntax:

• -o temperature 0.8: The temperature of the model. Increasing the temperature will make the model answer more creatively.

Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd llm-ollama
python3 -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

llm install -e '.[test]'

To run the tests:

pytest