tqdb 0.1.0

Embedded vector database using the TurboQuant algorithm (arXiv:2504.19874) — zero training, 2-4 bit compression, fast inner-product search

TurboQuantDB

An embedded vector database written in Rust with Python bindings, implementing the TurboQuant algorithm (arXiv:2504.19874) — zero training time, 2–4 bit compression, and provably unbiased inner product estimation.

Goal: make massive embedding datasets practical on lightweight hardware. A 50k-vector, 1536-dim collection that would occupy 293 MB as raw float32 fits in 70 MB on disk and 488 MB of RAM with TQDB b=4 — enabling laptop-scale RAG over millions of documents without a dedicated server.

Two deployment modes:

• Embedded — tqdb Python package (pip install tqdb), runs in-process (no daemon)
• Server — Axum HTTP service in server/, with multi-tenancy, RBAC, quotas, and async jobs

---

Key Properties

• Zero training — No train() step. Vectors are quantized and stored immediately on insert.
• 4.2× compression — Reduces float32 embeddings to 2–4 bits per coordinate; b=4 stores each vector in 1,466 bytes vs 6,144 bytes for float32.
• Unbiased scoring — QJL transform guarantees unbiased inner product estimation.
• Optional ANN index — Build an HNSW graph after loading data for fast approximate search.
• Metadata filtering — MongoDB-style filter operators on any metadata field.
• Crash recovery — Write-ahead log (WAL) ensures durability without explicit flushing.
• Python native — Built with PyO3 and Maturin; no server or sidecar required.

---

Installation

Prerequisites

• Rust stable toolchain
• Python 3.10+
• C++ compiler: Visual Studio Build Tools (Windows) · xcode-select --install (macOS) · build-essential (Linux)

Build from source

python -m venv venv
source venv/bin/activate        # Windows: .\venv\Scripts\activate
pip install maturin
maturin develop --release

Install pre-built wheel

pip install tqdb

---

Recommended Setup

Three presets covering the main use cases — pick one and you're ready:

from turboquantdb import Database

# High Quality — best recall (~97% at 50k×1536)
db = Database.open(path, dimension=DIM, bits=8, rerank=True)
db.create_index(max_degree=32, ef_construction=200, n_refinements=8)
results = db.search(query, top_k=10, ann_search_list_size=200)

# Balanced — recommended default (~89% recall, 5.7× smaller than float32)
db = Database.open(path, dimension=DIM, bits=4, rerank=True)
db.create_index(max_degree=32, ef_construction=200, n_refinements=5)
results = db.search(query, top_k=10, ann_search_list_size=128)

# Fast Build — fastest ingest (~83% recall, same disk as Balanced)
db = Database.open(path, dimension=DIM, bits=4, fast_mode=True, rerank=False)
db.create_index(max_degree=32, ef_construction=200, n_refinements=5)
results = db.search(query, top_k=10, ann_search_list_size=128)

Full parameter reference: docs/PYTHON_API.md

---

Quick Start

import numpy as np
from turboquantdb import Database

db = Database.open("./my_db", dimension=1536, bits=4, metric="ip", rerank=True)

db.insert("doc-1", np.random.randn(1536).astype("f4"), metadata={"topic": "ml"}, document="Machine learning intro")
db.insert("doc-2", np.random.randn(1536).astype("f4"), metadata={"topic": "systems"}, document="Rust memory model")

results = db.search(np.random.randn(1536).astype("f4"), top_k=5)
for r in results:
    print(r["id"], r["score"], r["document"])

---

Python API

Full reference: docs/PYTHON_API.md

# Open / create
db = Database.open(path, dimension, bits=4, seed=42, metric="ip",
                   rerank=True, fast_mode=False, rerank_precision=None,
                   collection=None)   # collection= → opens path/collection/

# Write
db.insert(id, vector, metadata=None, document=None)
db.insert_batch(ids, vectors, metadatas=None, documents=None, mode="insert")  # "insert"|"upsert"|"update"
db.upsert(id, vector, metadata=None, document=None)
db.update(id, vector, metadata=None, document=None)        # RuntimeError if not found
db.update_metadata(id, metadata=None, document=None)       # RuntimeError if not found

# Delete & retrieve
db.delete(id)                        # → bool
db.delete_batch(ids)                 # → int (count deleted)
db.get(id)                           # → {id, metadata, document} | None
db.get_many(ids)                     # → list[dict | None]
db.list_all()                        # → list[str]
db.list_ids(where_filter=None, limit=None, offset=0)       # paginated
db.count(filter=None)                # → int
db.stats()                           # → dict
len(db) / "id" in db                 # container protocol

# Search
results = db.search(query, top_k=10, filter=None, _use_ann=True,
                    ann_search_list_size=None, include=None)
# include: list of "id"|"score"|"metadata"|"document" (default all)

all_results = db.query(query_embeddings, n_results=10, where_filter=None)
# query_embeddings: np.ndarray (N, D) — returns list[list[dict]]

# Index
db.create_index(max_degree=32, ef_construction=200, n_refinements=5,
                search_list_size=128, alpha=1.2)

# Metadata filter operators
# $eq $ne $gt $gte $lt $lte $in $nin $exists $contains $and $or
db.search(query, top_k=5, filter={"year": {"$gte": 2023}})
db.search(query, top_k=5, filter={"$and": [{"topic": "ml"}, {"year": {"$gte": 2023}}]})

---

Recommended Presets

High Quality — recall matters most

db = Database.open(path, dimension=DIM, bits=8, rerank=True)
db.create_index(max_degree=32, ef_construction=200, n_refinements=8)
results = db.search(query, top_k=10, ann_search_list_size=200)
# ~97% Recall@10 at 50k×1536  |  ~38s ingest  |  119 MB disk

Balanced — default recommendation

db = Database.open(path, dimension=DIM, bits=4, rerank=True)
db.create_index(max_degree=32, ef_construction=200, n_refinements=5)
results = db.search(query, top_k=10, ann_search_list_size=128)
# ~89% Recall@10 at 50k×1536  |  ~26s ingest  |  70 MB disk

Fast Build — ingest speed is priority

db = Database.open(path, dimension=DIM, bits=4, fast_mode=True, rerank=False)
db.create_index(max_degree=32, ef_construction=200, n_refinements=5)
results = db.search(query, top_k=10, ann_search_list_size=128)
# ~83% Recall@10 at 50k×1536  |  ~22s ingest  |  70 MB disk

---

Benchmarks

Full results: BENCHMARKS.md

Highlights (50k × 1536, top_k=10, DBpedia OpenAI embeddings):

Engine	Ingest	Disk	RAM	p50	Recall@10
Engine-A (HNSW)	30.6s	398 MB	865 MB	1.73ms	99.75%
Engine-B (IVF-PQ)	77.6s	318 MB	526 MB	9.17ms	79.50%
TQDB b=8 HQ	59.2s	119 MB	537 MB	12.42ms	97.25%
TQDB b=4 Balanced	37.8s	70 MB	488 MB	9.98ms	89.15%
TQDB b=4 FastBuild	28.2s	70 MB	487 MB	5.30ms	83.35%

Engines identified by index algorithm. Reproduction scripts in benchmarks/.

TQDB b=4 is 5.7× smaller than Engine-A and uses 44% less RAM, within 11pp of recall.

---

RAG Integration

from turboquantdb.rag import TurboQuantRetriever

retriever = TurboQuantRetriever(db_path="./rag_db", dimension=1536, bits=4)
retriever.add_texts(texts=texts, embeddings=embeddings, metadatas=metadatas)

results = retriever.similarity_search(query_embedding=query_vec, k=5)
for r in results:
    print(r["score"], r["text"])

---

Architecture

TurboQuantDB is an embedded database — it runs in-process with no daemon.

./my_db/
├── manifest.json        — DB config (dimension, bits, seed, metric)
├── quantizer.bin        — Serialized quantizer state
├── live_codes.bin       — Memory-mapped quantized vectors (hot path)
├── live_vectors.bin     — Raw vectors for exact reranking (only if rerank_precision="f16" or "f32")
├── wal.log              — Write-ahead log
├── metadata.bin         — Per-vector metadata and documents
├── live_ids.bin         — ID → slot index
├── graph.bin            — HNSW adjacency list (if index built)
└── seg-XXXXXXXX.bin     — Immutable flushed segment files

Write path: insert() → quantize (QR rotation → MSE → Gaussian QJL) → WAL → live_codes.bin → flush to segment

Search (brute-force): query → precompute lookup tables → score all live vectors → top-k

Search (ANN): query → HNSW beam search → rerank → top-k

Quantization: Two-stage pipeline:

1. MSE — QR rotation + Lloyd-Max scalar quantization to bits per coordinate
2. QJL — Dense Gaussian projection, 1-bit quantized, bit-packed

The combination gives unbiased inner product estimates with near-optimal distortion, requiring no training data.

What comes from the paper vs. what is added here

The TurboQuant paper contributes the quantization algorithm — how to compress vectors and estimate inner products accurately. Its experiments use flat (exhaustive) search: all database vectors are scored against every query using the LUT-based asymmetric scorer. The paper's "indexing time virtually zero" claim refers to the quantizer requiring no training data, not to graph construction.

From the paper: two-stage MSE + QJL quantization, QR rotation, Lloyd-Max codebook, asymmetric LUT scoring, unbiased inner product estimation.

Added by TurboQuantDB (not in the paper): WAL persistence, memory-mapped storage, metadata/documents, HNSW graph index, reranking, Python bindings, and the HTTP server.

The brute-force search path (_use_ann=False) is the paper-conformant mode — it scores all vectors using TurboQuant's LUT scorer, matching the paper's experimental setup exactly. The HNSW index is a practical engineering addition that reduces the candidate set before scoring, enabling sub-linear search at the cost of approximate recall.

Module Map

Path	Responsibility
src/python/mod.rs	Database class — Python-facing API
src/storage/engine.rs	TurboQuantEngine — insert/search/delete orchestration
src/storage/wal.rs	Write-ahead log
src/storage/segment.rs	Immutable append-only segments
src/storage/live_codes.rs	Memory-mapped hot vector cache
src/storage/graph.rs	HNSW graph index
src/quantizer/prod.rs	ProdQuantizer — MSE + QJL orchestrator
src/quantizer/mse.rs	MseQuantizer — QR rotation + Lloyd-Max codebook
src/quantizer/qjl.rs	QjlQuantizer — 1-bit Gaussian projection, bit-packed
python/turboquantdb/rag.py	TurboQuantRetriever — LangChain-style wrapper
server/	Optional Axum HTTP service (separate Cargo workspace)

---

Server Mode

Status: experimental. The server crate compiles and the core endpoints work, but it has not been hardened for production use. The embedded library (tqdb Python package, from turboquantdb import Database) is the primary supported interface.

An optional Axum-based HTTP server is available in server/ for multi-tenant deployments. It adds API key authentication, quota enforcement, and async job management (compaction, index building, snapshots).

cd server && cargo build --release
TQ_SERVER_ADDR=0.0.0.0:8080 TQ_LOCAL_ROOT=./data ./target/release/turboquantdb-server

See server/README.md for the full endpoint reference. Key env vars:

Variable	Default	Description
TQ_SERVER_ADDR	127.0.0.1:8080	Bind address
TQ_LOCAL_ROOT	./data	Storage root
TQ_JOB_WORKERS	2	Async job thread count

---

Performance Roadmap

The current implementation already uses AVX2 SIMD for FWHT, the MSE centroid scan, and the QJL bit-unpack inner product.

GPU acceleration — batch ingest would benefit from cuBLAS GEMM (~3–5× for large batches on high-end cards). The ANN search path is memory-bound, not compute-bound, so GPU benefit there is minimal; the bottleneck is random cache misses during HNSW graph traversal rather than floating-point throughput.

AVX-512 codebook scan — on modern Intel CPUs the MSE centroid lookup can be vectorised 2× wider with AVX-512, potentially halving scoring latency per batch.

Persistent HNSW — incremental graph updates (no full rebuild after each ingest batch) would allow streaming use cases without periodic create_index() calls.

---

Research Basis

This is an independent implementation of ideas from the TurboQuant paper. The algorithm itself was authored by the original researchers.

Zandieh, A., Daliri, M., Hadian, M., & Mirrokni, V. (2025). TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate. arXiv:2504.19874

@article{zandieh2025turboquant,
  title={TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate},
  author={Zandieh, Amir and Daliri, Majid and Hadian, Majid and Mirrokni, Vahab},
  journal={arXiv preprint arXiv:2504.19874},
  year={2025}
}

---

License

Apache License 2.0 — see LICENSE.