Run embedding models using ONNX
Project description
llm-embed-onnx
Run embedding models using ONNX
This LLM plugin is a wrapper around onnx_embedding_models by Benjamin Anderson.
Installation
Install this plugin in the same environment as LLM.
llm install llm-embed-onnx
Usage
This plugin adds the following embedding models, which can be listed using llm embed-models
:
onnx-bge-micro
onnx-gte-tiny
onnx-minilm-l6
onnx-minilm-l12
onnx-bge-small
onnx-bge-base
onnx-bge-large
You can run any of these models using llm embed
command:
llm embed -m onnx-bge-micro -c "Example content"
This will output a 384 length JSON array of floating point numbers, starting:
[-0.03910085942622519, -0.0030843335461659795, 0.032797761260860724,
The first time you use any of these models the model will be downloaded to the llm_embed_onnx
directory in your LLM data directory. On macOS this defaults to:
~/Library/Application Support/io.datasette.llm/llm_embed_onnx
For more on how to use these embedding models see the LLM embeddings documentation.
Development
To set up this plugin locally, first checkout the code. Then create a new virtual environment:
cd llm-embed-onnx
python3 -m venv venv
source venv/bin/activate
Now install the dependencies and test dependencies:
llm install -e '.[test]'
To run the tests:
pytest
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for llm_embed_onnx-0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 946a9694f046f09965e12d481220dc8a146b0f6bbabe5f37457ebe2b2d4431f0 |
|
MD5 | 8ff993d7018c5df9fd481384f9397ec7 |
|
BLAKE2b-256 | 4346f2c5df1d94e783874aa5db1bfbb80e88893dde198377a4fe501999baeec5 |