llm-embed-ollama

LLM plugin providing access to embedding models running on local Ollama server.

Installation

Install this plugin in the same environment as LLM.

llm install llm-embed-ollama

Background

Ollama provides Few embedding models. This plugin enables the usage of those models using llm and ollama embeddings..

To utilize these models, you need to have an instance of the Ollama server running.

See also Embeddings: What they are and why they matter for background on embeddings and an explanation of the LLM embeddings tool.

See also Ollama Embeddings Models Blog

Usage

This plugin adds support for the following embedding models available in ollama:

• all-minilm
• nomic-embed-text
• mxbai-embed-large
• bge-large: Embedding model from BAAI mapping texts to vectors.
• bge-m3: BGE-M3 is a new model from BAAI distinguished for its versatility in Multi-Functionality, Multi-Linguality, and Multi-Granularity.

The models needs to be downloaded. Using `ollama pull the first time you try to use them.

See the LLM documentation for everything you can do.

To get started embedding a single string, run the following:

Make sure you have the appropriate ollama model.

ollama pull all-minilm

llm embed -m all-minilm -c 'Hello world'

This will output a JSON array of 384 floating point numbers to your terminal.

To calculate and store embeddings for every README in the current directory (try this somewhere with a node_modules directory to get lots of READMEs) run this:

llm embed-multi ollama-readmes \
    -m all-minilm \
    --files . '**/README.md' --store

Then you can run searches against them like this:

llm similar ollama-readmes -c 'utility functions'

Add | jq to pipe it through jq for pretty-printed output, or | jq .id to just see the matching filenames.

Prefix and suffix support

Some embedding models expect prefixed or suffixed input (for example, instruction-style or query embeddings).

This plugin supports optional embedding prefixes and suffixes provided by llm. When a prefix or suffix is set on the embedding model, it is automatically applied before generating embeddings.

Example (Python):

import llm

model = llm.get_embedding_model("all-minilm")
model.prefix = "query: "

embedding = model.embed("hello world")

This is useful for models that distinguish between query and document embeddings.

Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd llm-embed-ollama
python3 -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

llm install -e '.[test]'

To run the tests:

pytest