tqvs

Composable vector store with pluggable persistence, dtype quantization, and similarity search.

tqvs gives you a keyed vector store that works entirely in-process — no server, no network calls. Vectors are quantized on the fly to the storage dtype you choose, persisted through a swappable backend, and queried with brute-force similarity search (with optional CUDA acceleration via PyTorch).

Installation

pip install tqvs              # core (NumPy only, includes the npy backend)
pip install tqvs[hdf5]        # + HDF5 backend  (h5py)
pip install tqvs[lmdb]        # + LMDB backend  (lmdb)
pip install tqvs[lance]       # + Lance backend (pylance)
pip install tqvs[parquet]     # + Parquet backend (pyarrow)
pip install tqvs[torch]       # + GPU-accelerated metrics
pip install tqvs[all]         # all optional backends

Requires Python 3.12+.

Quick start

import numpy as np
from tqvs import create_vector_store, StoreDtype

store = create_vector_store("./my_store", dim=768, dtype=StoreDtype.INT8_SYM)

store.add("doc/1", np.random.rand(768).astype(np.float32), metadata={"title": "Hello"})
store.add("doc/2", np.random.rand(768).astype(np.float32))

results = store.query(np.random.rand(768).astype(np.float32), k=5)
for r in results:
    print(r.key, r.score)

store.save()

Fluent builder

from tqvs import VectorStoreBuilder, StoreDtype, Hdf5Backend, LoadMode

store = (
    VectorStoreBuilder()
    .at("./my_store")
    .with_dim(768)
    .with_dtype(StoreDtype.FLOAT16)
    .with_backend(Hdf5Backend())
    .with_load_mode(LoadMode.MMAP)
    .build()
)

Batch insert

keys = [f"vec/{i}" for i in range(10_000)]
vectors = np.random.rand(10_000, 768).astype(np.float32)
store.add_many(keys, vectors)

Prefix-filtered queries

# Search only keys starting with "doc/"
results = store.query(q, k=10, prefix="doc/")

# Iterate matching keys
for key in store.keys(prefix="img/"):
    print(key)

Storage dtypes

Vectors are quantized at insert time and dequantized at query time. Smaller dtypes reduce memory and disk usage at the cost of some precision.

TurboQuant dtypes apply a random rotation before quantization to spread quantization error more evenly across dimensions.

Persistence backends

Five backends are available. They all share the same in-memory store — the backend only determines how data is serialized to and from disk.

Performance comparison

Benchmarked with 1,000,000 128-dimensional vectors, averaged across all 11 storage dtypes. In-memory operations (add, query, key lookup) are backend-independent since the backend is only involved during save/reload.

Save throughput (vectors/sec, 1M vectors)

Reload throughput (vectors/sec, 1M vectors)

In-memory operations (all backends equivalent)

Choosing a backend

• HDF5 — Best all-round choice. Fastest saves, near-fastest reloads. Single .h5 file is easy to manage. Good default for most workloads.
• NumPy (npy) — Zero optional dependencies. Fastest reloads, solid saves. Choose this if you want to avoid extra packages.
• LMDB — Comparable I/O to HDF5/NumPy, and supports transactional semantics. Useful when concurrent read access matters.
• Lance — Slightly slower reloads than the top tier but still competitive. Best suited when you already use the Lance ecosystem and want a unified format.
• Parquet — Roughly 10× slower saves and 3–4× slower reloads than HDF5. Choose only when interop with Spark, DuckDB, Pandas, or other columnar tools is a priority.

Dtype impact on I/O

Smaller quantized dtypes save and reload faster because there is less data to write and read:

(1M vectors, dim=128)

Disk usage

On-disk size depends primarily on the storage dtype and secondarily on the backend format. NpyBackend and Hdf5Backend are the most space-efficient, storing close to the raw vector data with minimal overhead. LMDB adds ~2× overhead due to its B-tree page structure. Lance and Parquet add modest columnar metadata.

(100K vectors, dim=128)

Using TURBO_2BIT instead of FLOAT32 reduces on-disk size by ~11× (50 MB → 4.6 MB per 100K vectors). LMDB is consistently ~2× larger than the other backends due to its B-tree page overhead, but this is the cost of its transactional and concurrent-read capabilities.

TurboQuant performance overhead

TurboQuant dtypes apply a random orthogonal rotation before scalar quantization, which spreads quantization error evenly but adds computational cost. The overhead is most visible during queries, where dequantization sits in the hot loop.

Insert overhead (add_many, 1M vectors)

TurboQuant inserts are only marginally slower than standard quantization at the same bit-width — the rotation matrix multiply is amortized across the batch:

Query overhead (query_prefix, 1M vectors)

Queries are significantly slower because each scored vector requires unpacking, codebook lookup, and inverse rotation:

TURBO_2BIT queries at ~0.30× FLOAT32 speed — reasonable given the 11× storage reduction. TURBO_3BIT and TURBO_4BIT are notably slower because the sub-byte unpacking for 3- and 4-bit codes is more complex. If query speed matters more than maximum compression, prefer INT4 or INT8_SYM over the higher-bit TurboQuant variants.

Similarity metrics

Three built-in metrics are provided:

• cosine_similarity — default, higher = more similar
• dot_product — raw dot product
• euclidean_distance — negative L2 distance (higher = closer)

from tqvs import create_vector_store, euclidean_distance

store = create_vector_store("./store", dim=128, metric=euclidean_distance)

GPU acceleration with PyTorch

Install the torch extra to unlock CUDA-accelerated similarity scoring:

pip install tqvs[torch]

When device is set, query() and score() automatically dispatch to PyTorch equivalents of the three built-in metrics. Vectors are transferred to the target device, scored using torch tensor ops, and results are returned as NumPy arrays.

store = create_vector_store("./store", dim=768, device="cuda")

# query() and score() now run on GPU
results = store.query(q, k=10)

This is most beneficial when:

• You use cosine_similarity (the default) — the normalization step adds enough compute to offset the CPU→GPU→CPU data transfer overhead. For dot_product and euclidean_distance, NumPy is already fast enough that the transfer cost makes CUDA slower in practice.
• Vector count is large (10k+) — the cost of host-to-device transfer is amortized over a larger scoring pass.
• Dimensionality is high (768, 1024, 1536) — GPU parallelism shines on wide matrix operations.

CUDA speedup (cosine_similarity, RTX 5090)

For dot_product and euclidean_distance, CUDA is 0.1–0.9× (slower than NumPy) across all tested configurations because the operations are dominated by a single matrix-vector product that NumPy (MKL/OpenBLAS) handles efficiently without data transfer overhead.

Recommendation: Use device="cuda" with the default cosine_similarity metric when you have 10k+ vectors at dim ≥ 768. For other metrics or smaller stores, leave device unset.

For small stores (under ~10k vectors) or low dimensions, NumPy on CPU is already fast enough and the data-transfer overhead makes GPU dispatch slower. The device parameter also accepts "cpu" to use PyTorch's CPU kernels (useful for MKL-backed PyTorch builds), though in most cases plain NumPy is comparable.

Note: PyTorch acceleration applies only to the scoring step of query() and score(). Insert, delete, save, reload, and key operations are always NumPy-based and unaffected by the device setting.

Load modes

License

See pyproject.toml for package metadata.