turboqvec-rs

Compressed vector store for Python, backed by Rust. Store large float arrays at 8–15x smaller footprint and run cosine similarity queries directly on the compressed form — no decompression needed.

Built on PolarQuant (TurboQuant Stage 1, Google Research, ICLR 2026). Rust core with PyO3 bindings and Rayon parallel scan.

Install

pip install turboqvec

Build from source:

git clone https://github.com/Manojython/turboqvec-rs
cd turboqvec-rs
pip install maturin
maturin develop --release

Usage

from turboqvec import CompressedVectorStore

store = CompressedVectorStore(dim=384, bits=4)

# Insert — accepts lists or numpy arrays
store.insert(id=0, vector=my_embedding)
store.insert_batch([(id, vec) for id, vec in pairs])  # parallel, faster

# Query — returns (id, score) tuples sorted by cosine similarity
results = store.query(query_embedding, top_k=10)

# Stats
print(f"{len(store):,} vectors, {store.memory_bytes() / 1e6:.1f} MB compressed")

See notebooks/quickstart.ipynb for a full walkthrough with real embeddings.

Accuracy

Validated on 500 BGE-small-en-v1.5 sentence embeddings (d=384, 20newsgroups).

Pearson/Spearman correlation of scores vs true cosine: 0.998 at 4-bit.

PolarQuant slightly underestimates cosine similarity (mean drift ≈ -0.003 at 4-bit). Rank order is preserved so top-k retrieval is not affected. For absolute threshold queries (e.g. score > 0.7), add ~0.003 as a calibration offset.

Storage

How it works

Ingestion

Each vector goes through: norm → normalise → rotate (R·v̂) → Lloyd-Max quantize → bit-pack. The rotation matrix R is a fixed random orthogonal matrix (QR decomp) that decorrelates all dimensions so a single codebook fits every coordinate position. Only the norm (4 bytes exact) and the packed bin indices are stored.

Retrieval

For a query q, we build a lookup table once: lut[i][b] = (R·q)[i] × centroid[b]. Then for each database vector, the score is a sum of d table lookups plus one multiply — no matrix math per vector. Rayon parallelises the scan across all N entries.

This is asymmetric distance computation (the same idea FAISS PQ uses). One matrix multiply per query regardless of N.

Why not QJL (Stage 2)?

TurboQuant adds a 1-bit residual sketch for KV cache attention vectors. For normalised sentence embeddings the Stage 1 bias is already near zero — adding QJL introduced noise and dropped Pearson correlation from 0.996 to 0.48. Stage 1 alone is the right primitive for standard embeddings.

Repo layout

turboqvec-rs/
├── src/               Rust source
│   ├── codebook.rs    Lloyd-Max quantizer, global cache
│   ├── rotation.rs    QR decomposition, global cache
│   ├── encode.rs      encode pipeline + bit packing + Rayon batch
│   ├── decode.rs      decode pipeline (for reconstruction)
│   ├── similarity.rs  asymmetric LUT scoring
│   ├── store.rs       CompressedVectorStore
│   └── py_bindings.rs PyO3 wrapper
├── python/turboqvec/  Python package
├── notebooks/         Quickstart notebook
├── benchmarks/        Comparison scripts vs fp32 baseline
└── tests/             Rust integration tests

Related

• fastrustrag — MinHash deduplication for RAG pipelines
• pageindex-rs — Hierarchical document indexing for LLMs
• TurboQuant paper — Google Research, ICLR 2026

License

MIT