emb-opt 0.0.2

A lightweight framework to efficiently screen vector databases

emb_opt

emb_opt uses reinforcement learning to efficiently find items in a vector database that maximize some score_function

See the documentation site for documentation and tutorials

Install

pip install emb_opt

Supported Backends

The following vector databases are supported

FAISS

from emb_opt.backends.faiss import FaissDatabase
vector_database = FaissDatabase(my_faiss_index, k, faiss_search_params)

Huggingface Datasets

from emb_opt.backends.hf import HFDatabase
my_dataset.add_faiss_index(embedding_column_name)
vector_database = HFDatabase(my_dataset, embedding_column_name, k)

Qdrant

from emb_opt.backends.qdrant import QdrantDatabase
vector_database = QdrantDatabase(my_qdrant_client, collection_name, k, qdrant_search_params)

Chroma

from emb_opt.backends.chroma import ChromaDatabase
vector_database = ChromaDatabase(my_chroma_collection, k, chroma_query_kwargs)

Basic Usage

For more detail on the API, see the basic tutorial

from emb_opt.core import Score
from emb_opt.runner import Runner
from emb_opt.query_update import RLUpdate

score = Score(my_score_function)
query_update = RLUpdate(lr=0.5)
runner = Runner(vector_database, score, query_update)

init_query_vectors = ... # any function to initalize query vectors
iterations = 15

results_log = runner.search(init_query_vectors, iterations)