A lightweight framework to efficiently screen vector databases
Project description
emb_opt
emb_opt
uses reinforcement learning to efficiently find items in a
vector database that maximize some score_function
See the documentation site for documentation and tutorials
Install
pip install emb_opt
Supported Backends
The following vector databases are supported
FAISS
from emb_opt.backends.faiss import FaissDatabase
vector_database = FaissDatabase(my_faiss_index, k, faiss_search_params)
Huggingface Datasets
from emb_opt.backends.hf import HFDatabase
my_dataset.add_faiss_index(embedding_column_name)
vector_database = HFDatabase(my_dataset, embedding_column_name, k)
Qdrant
from emb_opt.backends.qdrant import QdrantDatabase
vector_database = QdrantDatabase(my_qdrant_client, collection_name, k, qdrant_search_params)
Chroma
from emb_opt.backends.chroma import ChromaDatabase
vector_database = ChromaDatabase(my_chroma_collection, k, chroma_query_kwargs)
Basic Usage
For more detail on the API, see the basic tutorial
from emb_opt.core import Score
from emb_opt.runner import Runner
from emb_opt.query_update import RLUpdate
score = Score(my_score_function)
query_update = RLUpdate(lr=0.5)
runner = Runner(vector_database, score, query_update)
init_query_vectors = ... # any function to initalize query vectors
iterations = 15
results_log = runner.search(init_query_vectors, iterations)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
emb_opt-0.0.2.tar.gz
(14.3 kB
view hashes)
Built Distribution
emb_opt-0.0.2-py3-none-any.whl
(16.3 kB
view hashes)