flexinfer-vector-utils

Unified vector storage and embedding utilities for FlexInfer services.

Features

• Hybrid Embedder: Automatic fallback from OpenAI/vLLM to local sentence-transformers models.
• Qdrant Storage: Generalized async client for Qdrant collection management and search.
• Batching: Optimized batch processing for high-throughput embedding generation.

Usage

Embeddings

from vector_utils.embeddings import HybridEmbedder

# Tries OpenAI -> vLLM -> Local fallback
embedder = HybridEmbedder(
    openai_api_key="...",
    enable_local_fallback=True
)

embeddings = await embedder.embed_batch(["text1", "text2"])

Vector Store

from vector_utils.store import QdrantStore, VectorItem

store = QdrantStore(url="http://qdrant:6333", collection_name="docs")
await store.initialize()

await store.upsert([
    VectorItem(id="1", vector=[0.1, ...], payload={"type": "doc"})
])

Installation

pip install flexinfer-vector-utils
# With local fallback support
pip install "flexinfer-vector-utils[local]"