Fast, light, accurate library built for retrieval embedding generation
Project description
⚡️ What is FastEmbed?
FastEmbed is an easy to use -- lightweight, fast, Python library built for retrieval embedding generation.
The default embedding supports "query" and "passage" prefixes for the input text. The default model is Flag Embedding, which is top of the MTEB leaderboard. Here is an example for Retrieval Embedding Generation
-
Light
- Quantized model weights
- ONNX Runtime for inference
- No hidden dependencies on PyTorch or TensorFlow via Huggingface Transformers
-
Accuracy/Recall
- Better than OpenAI Ada-002
- Default is Flag Embedding, which is top of the MTEB leaderboard
-
Fast
- About 2x faster than Huggingface (PyTorch) transformers on single queries
- Lot faster for batches!
- ONNX Runtime allows you to use dedicated runtimes for even higher throughput and lower latency
🚀 Installation
To install the FastEmbed library, pip works:
pip install fastembed
📖 Usage
from fastembed.embedding import FlagEmbedding as Embedding
documents: List[str] = [
"passage: Hello, World!",
"query: Hello, World!", # these are two different embedding
"passage: This is an example passage.",
# You can leave out the prefix but it's recommended
"fastembed is supported by and maintained by Qdrant."
]
embedding_model = Embedding(model_name="BAAI/bge-base-en", max_length=512)
embeddings: List[np.ndarray] = list(embedding_model.embed(documents))
Why fast?
It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:
- Quantized model weights
- ONNX Runtime which allows for fast inference on CPU and other dedicated runtimes
Why light?
- No hidden dependencies on PyTorch or TensorFlow via Huggingface Transformers
- We do use the tokenizer from Huggingface Transformers, but it's a light dependency
Why accurate?
- Better than OpenAI Ada-002
- Top of the Embedding leaderboards e.g. MTEB
Similar Work
Ilyas M. wrote about using FlagEmbeddings with Optimum over CUDA.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
fastembed-0.0.5a1.tar.gz
(9.4 kB
view hashes)
Built Distribution
Close
Hashes for fastembed-0.0.5a1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5dd57938d076976c44684aa78df79f92ac3be9888fe5e6c24da1c32aab3207a1 |
|
MD5 | 86d3a2f95e6e93e7c50acad33514985f |
|
BLAKE2b-256 | 391f41e3fbfacf1e34a9fdc869504f4d5a39d4c4dc08eac4e32bed128f52fa3b |