rixdb 0.1.1.dev0

Fast, embeddable C++ vector database with Python bindings

RixDB

Fast, embeddable vector database for Python — written in C++.

⚠️ Pre-implementation. RixDB is in active design — no C++ source exists yet. The code samples and API below reflect the planned design, not a working package. Follow the repo or watch releases to be notified when the first alpha is available.

RixDB is intended to be a zero-dependency, SIMD-accelerated vector index with memory-mapped persistence and a clean Python API. It is designed to be embedded directly inside your application — no server, no daemon, no configuration overhead.

# Planned API — not yet functional
import rix
 
db = rix.Index.open_or_create(dim=768, metric="cosine", path="./index")
db.upsert(ids, vectors, metadata)
results = db.search(query_vector, top_k=8)

RixDB is a ground-up implementation of HNSW-based vector search, built to understand the internals rather than wrap an existing library. It prioritizes correctness, clean API design, and the specific feature set needed for local embedding retrieval — not raw benchmark numbers.

---

Why RixDB

Most local vector search options make you choose between simplicity and performance. ChromaDB is simple but slow on large corpora. FAISS is fast but has no metadata, no incremental updates, and a difficult API. hnswlib is close but general-purpose and lacks a metadata store.

RixDB is built for the specific case of embedding-based retrieval inside a single Python process:

• Zero runtime dependencies. Will ship as a single compiled .so / .pyd. Nothing to install beyond pip install rix once released.
• Fast cold load (design target). The index is planned to be memory-mapped so the OS page cache handles most I/O, with load time largely independent of index size.
• SIMD-accelerated queries (planned). Support for AVX-512, AVX2, SSE4.2, and NEON is intended, with code paths selected automatically at runtime based on CPU features.
• Incremental updates (planned). The index is intended to support inserts and deletes without full rebuilds, optimized for indices that change frequently.
• Metadata-native (design goal). The design calls for every vector to carry structured metadata stored in a flat binary layout alongside the index, avoiding a separate lookup table.

---

Installation

Package name vs import name: The PyPI distribution name is rixdb (what you type in pip install). The Python module name is rix (what you type in import rix). This follows the same convention as other libraries — pip install Pillow / import PIL, pip install scikit-learn / import sklearn.

Planned install (Phase E — production release)

pip install rixdb

Pre-built wheels are planned for:

Platform	Architecture
Linux	x86_64, aarch64
macOS	x86_64, arm64
Windows	x86_64

To build from source (once C++ implementation exists), a C++17 compiler and CMake 3.21+ will be required.

---

Quick Start

Not yet functional. The following shows the planned API once Phase A is complete. The code samples reflect the final design, not the current state of the repository.

Create and populate an index

import rix
import numpy as np

# Open or create a persistent index
db = rix.Index.open_or_create(
    dim=768,
    metric="cosine",
    path="./my_index"
)

# Generate some example vectors (replace with real embeddings)
ids = ["doc_0", "doc_1", "doc_2"]
vectors = np.random.rand(3, 768).astype(np.float32)
metadata = [
    {"source": "readme.md", "section": "intro"},
    {"source": "api.md",    "section": "usage"},
    {"source": "guide.md",  "section": "install"},
]

db.upsert(ids, vectors, metadata)

Search

query = np.random.rand(768).astype(np.float32)
results = db.search(query, top_k=3)

for r in results:
    print(r.id, r.score, r.metadata)

Update and delete

# Upsert replaces existing IDs automatically
db.upsert(["doc_0"], new_vector, new_metadata)

# Delete by ID
db.delete(["doc_1"])

Use as a context manager

with rix.Index.open("./my_index") as db:
    results = db.search(query_vector, top_k=5)

---

API Overview

rix.Index

Method	Description
Index.create(dim, metric, path, params)	Create a new empty index
Index.open(path)	Open an existing index
Index.open_or_create(dim, metric, path, params)	Open if exists, create if not
upsert(ids, vectors, metadata)	Insert or replace vectors by ID
search(vector, top_k, ef_search, filters)	Approximate nearest neighbor search
search_batch(vectors, top_k, ef_search)	Parallel batch search, one result list per query
delete(ids)	Remove vectors by ID
get(ids)	Fetch metadata without ANN search
contains(id)	Check if an ID exists
list_ids(prefix)	List all stored IDs
compact()	Force compaction of deleted entries
stats()	Index statistics and file sizes
close()	Flush and close (called automatically)

rix.SearchResult

Field	Type	Description
id	str	External ID provided at insert time
score	float	Similarity score (higher = more similar)
metadata	dict	All metadata fields stored at insert time

Metrics

Value	Distance Function	Use Case
"cosine"	Cosine similarity	Text and code embeddings (default)
"l2"	Euclidean distance	Image embeddings, some ML models
"ip"	Inner product	Recommendation systems

---

Tuning

RixDB works well at default settings for most use cases. The key parameters are:

Parameter	Default	Effect
M	16	Connections per node. Higher = better recall, more memory
ef_construction	200	Build quality. Higher = better recall, slower build
ef_search	50	Query candidate pool. Higher = better recall, slower query

Pass custom parameters at creation time:

db = rix.Index.create(
    dim=768,
    metric="cosine",
    path="./index",
    params={"M": 32, "ef_construction": 400}
)

Built-in profiles

# High recall — best accuracy, higher memory
params = rix.profiles.high_recall

# Low memory — reduced RAM, slightly lower recall
params = rix.profiles.low_memory

# Fast build — quicker indexing, default query performance
params = rix.profiles.fast_build

---

Benchmarks

Note — RixDB is not yet implemented. The numbers below are conservative design targets, not measured results. RixDB is an initial, first-principles implementation — these targets reflect what a correct, well-optimized implementation of HNSW with AVX2 SIMD and mmap persistence should be able to achieve, not what has been verified. Benchmarks will be measured and published after Phase B is complete. Until then, treat all numbers here as aspirational.

Design targets on a mid-range developer laptop (Intel Core i7, AVX2, no GPU). Index dimension: 768. Metric: cosine. Default parameters (M=16, ef_search=50).

Corpus Size	Query P95 (target)	Build Time (target)	Cold Load (target)	Recall@10 (target)
10K vectors	< 5ms	< 30s	< 100ms	≥ 0.95
100K vectors	< 20ms	< 5min	< 100ms	≥ 0.95
500K vectors	< 50ms	< 30min	< 100ms	≥ 0.95

Feature comparison with alternatives (competitor numbers are from their published benchmarks — RixDB column is targets only):

Library	Query P95	Cold Load	Incremental Updates	Metadata
RixDB (target)	< 20ms	< 100ms	✓	✓
ChromaDB	~25ms	~400ms	✓	✓
hnswlib	~4ms	~180ms	✗	✗
FAISS (IVF)	~3ms	~200ms	✗	✗

RixDB's query latency target is intentionally conservative. hnswlib is written by the authors of the HNSW paper and is extensively optimized. Matching it on the first implementation attempt is not a realistic expectation. The value RixDB adds over hnswlib is incremental updates and a built-in metadata store — not raw query speed.

---

Architecture

RixDB is built on three core components:

HNSW index — Hierarchical Navigable Small World graph for O(log n) approximate nearest neighbor search. Parameters M, ef_construction, and ef_search control the recall/speed/memory tradeoff.

SIMD distance kernel — Runtime CPU detection selects the best available path: AVX-512, AVX2, SSE4.2, NEON, or scalar fallback. Cosine similarity reduces to a dot product after L2 normalization at insert time.

mmap persistence — The index and metadata live in memory-mapped binary files. The OS page cache manages physical memory. Cold load time is independent of index size.

Full architecture documentation is in docs/architecture.md.

---

Building From Source

Not yet possible. No C++ source exists yet. The following describes the planned build process for when Phase A implementation begins.

Requirements (planned): C++17 compiler (GCC 10+, Clang 12+, MSVC 2019+), CMake 3.21+, Python 3.11+.

git clone https://github.com/Pranav1173/RixDB
cd RixDB
pip install -e ".[dev]"

To run the C++ unit tests:

cmake -B build -DRIX_BUILD_TESTS=ON
cmake --build build
./build/rix_test

To run the Python tests:

pytest tests/python/

---

Contributing

Contributions are welcome. Please read CONTRIBUTING.md before opening a pull request.

For significant changes — new features, API changes, algorithm modifications — open an issue first to discuss the design. The architecture document is the reference for all design decisions.

Areas where contributions are especially welcome:

• Additional SIMD kernels (AVX-512 path, improved NEON)
• Scalar quantization (int8 vector support)
• Windows ARM64 support
• Language bindings beyond Python

---

License

Apache 2.0 — see LICENSE.

---

Status

RixDB is currently in pre-implementation planning. The architecture is fully documented and the repository is set up, but no C++ code has been written yet. Development begins at Phase A — a correct scalar HNSW implementation with Python bindings.

Planned release milestones:

Phase A  →  0.1.0-alpha   Correct scalar HNSW + Python bindings
Phase B  →  0.2.0-alpha   SIMD + mmap + verified benchmarks
Phase C  →  0.3.0-alpha   Incremental updates + compaction
Phase D  →  0.4.0-beta    Parallel batch insert + thread safety
Phase E  →  1.0.0         PyPI publication + pre-built wheels

Follow the repository or watch releases to be notified when the first alpha is available.

Current version: 0.1.0.dev0 (name reservation only)