monavec 1.0.0

Ultra-low resource embedded vector search for edge AI

MonaVec

Embedded vector search kernel.
No server. No training. Runs anywhere.

Named after Monachus monachus — the Mediterranean monk seal.
One of Earth's rarest marine mammals. Quiet. Resilient. Built to last.
First product of Mona — embedded AI infrastructure.

---

AI is moving to the edge.
Vector search hasn't caught up.
MonaVec bridges that gap.

---

Why MonaVec Exists

Every vector search tool was built for servers. Qdrant needs a running service. FAISS requires gigabytes of RAM and a training pass. Weaviate is a cloud product.

Meanwhile, on-device LLMs are running on phones. Offline agents are being deployed in places with no internet. Industrial systems need semantic search with zero external dependencies.

MonaVec was built for that world. A single .mvec file. A library call. That's it.

No training pass, ever. FAISS IVF requires collecting a corpus and running k-means before the index is usable. MonaVec's quantization is data-oblivious — add the first vector, search immediately. Incremental data, memory-constrained devices, cold-start deployments: none of them require a training phase.

One file. One copy. The entire index — vectors, quantization parameters, rotation seed — lives in a single .mvec file. Bundle it in a mobile app, flash it to a device, ship it with an offline agent. No migrations. No daemon. No schema to keep in sync.

Deterministic by design. The ChaCha20 seed is embedded in the file. Load the same .mvec on any machine: the file decodes to the same index everywhere and search returns the same top-K results — byte-identical within a build, and reproducible across architectures (the heterogeneous SIMD kernels agree to within a validated 1e-4 tolerance). On a device with no observability, non-deterministic search behavior is nearly impossible to debug. MonaVec eliminates that class of problem.

Pure Rust, zero C dependencies. FAISS requires OpenBLAS or MKL. MonaVec's core has no non-Rust dependencies. Compile it for Android NDK, WASM, embedded RISC-V, Apple Silicon, Windows ARM — anywhere Rust runs, MonaVec runs.

	MonaVec	FAISS	Qdrant	Weaviate
Embedded (no server)	✅	✅	❌	❌
Edge / offline	✅	❌	❌	❌
Zero training	✅	❌	❌	❌
RAM for 1M × 1536-dim	768 MB	~6 GB	~6 GB	~6 GB

---

What It Is

MonaVec is a pure Rust library — a kernel, not a database. Embed it directly with a library call, or run monavec serve for a full REST API, admin UI, and CLI. The kernel works either way.

• 8–16× compression via data-oblivious 4-bit quantization (no training data needed)
• SIMD-accelerated search — AVX-512, AVX2, NEON, runtime dispatch
• Three index types: BruteForce, IvfFlat, HNSW
• Pre-filtered hybrid retrieval — dense + sparse (BM25 + SPLADE), filter applied before scoring
• Deterministic — same seed → same results, always
• Library or service — your choice — import it in Rust/Python, or run monavec serve for a REST API, web UI, and terminal CLI
• Ships as a Rust crate (monavec-core) and a Python package (monavec)

---

Install

Python

pip install monavec

Requires Python ≥ 3.9. NumPy is the only runtime dependency.

Rust

[dependencies]
monavec-core = "1.0"

# Optional index types
monavec-core = { version = "1.0", features = ["ivf", "hnsw"] }

---

Quick Start

Python

import numpy as np
from monavec import MonaVec

index = MonaVec(dim=384, metric="cosine", bit_width=4)

ids  = list(range(1000))
vecs = np.random.randn(1000, 384).astype(np.float32)
index.add(ids, vecs)

query   = np.random.randn(384).astype(np.float32)
results = index.search(query, k=10)
# [{"id": 42, "score": 0.94}, ...]

# Pre-filtered search — allowlist applied BEFORE scoring (no recall loss)
results = index.search(query, k=5, allowlist=list(range(0, 1000, 2)))

# HNSW — log(n) search, best for > 100K vectors
index = MonaVec(dim=384, metric="cosine", bit_width=4, index_type="hnsw")
index.add(ids, vecs)
index.build()
results = index.search(query, k=10)

# Persist
index.save("my_index.mvec")
index = MonaVec.load("my_index.mvec")

Rust

use monavec_core::{MonaVec, Config, Metric};

let config = Config::new(384, Metric::Cosine, 4);
let mut index = MonaVec::new(config)?;

index.add(&[0u64, 1, 2], &[vec![0.1f32; 384], vec![0.2f32; 384], vec![0.3f32; 384]])?;

let results = index.search(&vec![0.1f32; 384], 3, None)?;
let results = index.search(&vec![0.1f32; 384], 3, Some(&[0u64, 2]))?;

index.save("my_index.mvec")?;
let index = MonaVec::load("my_index.mvec")?;

---

Performance

AG News 45K × 1024-dim (BGE-M3 embeddings, cosine, i7-13620H, single-core, release + x86-64-v3):

System	recall@10	QPS	Mem
MonaVec BruteForce 4-bit	0.960	137	27 MB
MonaVec HNSW 4-bit	0.954	1,264	249 MB

Key result: MonaVec 4-bit BruteForce reaches the highest recall (0.960) in just 27 MB with zero configuration. Direct comparison against FAISS-IVF, usearch, and hnswlib is in vs Embedded Competitors below.

---

fashion-mnist 60K × 784-dim (raw pixels, L2, single-threaded):

System	recall@10	QPS	Notes
MonaVec BruteForce 4-bit + fit()	0.624	68	zero-training scalar quantization
MonaVec HNSW M=32 ef=40	0.614	1,315	19× faster than BF at same recall
MonaVec HNSW M=32 ef=400	0.624	308	matches BF recall
FAISS IVFPQ (trained)	~0.85+	—	trained codebook, not zero-training

Key result: With fit(), BruteForce recall improves 0.41 → 0.62 (+52%). HNSW M=32 delivers the same recall as BruteForce at 19× higher QPS.

Why the gap to FAISS IVFPQ (~0.85+)? FAISS IVFPQ requires a training pass: it runs k-means over the corpus to learn a data-specific codebook, then applies Product Quantization — jointly encoding groups of dimensions to exploit spatial correlation. MonaVec uses fixed Lloyd-Max tables (N(0,1), no training) and quantizes each dimension independently. On pixel data with high inter-dimension correlation, this is a known tradeoff. For semantic embeddings — MonaVec's primary target — the distribution is already well-conditioned and recall reaches 0.95+.

---

---

glove-100 1.18M × 100-dim (GloVe word embeddings, cosine, single-threaded):

System	recall@10	QPS	Notes
MonaVec BruteForce 4-bit	0.865	42	quantization ceiling at 100-dim
MonaVec HNSW M=32 ef=400	0.800	220	M=32 insufficient at 1M+ scale
MonaVec HNSW M=64 ef=200	0.831	232	auto-M threshold: N ≥ 1M → M=64
MonaVec HNSW M=64 ef=400	0.850	125	approaches BF ceiling

Key result: At N=1M+, M=32 yields suboptimal graph connectivity (recall 0.800). M=64 restores recall to 0.831 at identical QPS — same throughput, +3.1 pp recall. MonaVec automatically selects M based on corpus size: M=32 for N < 1M, M=64 for N ≥ 1M.

Why does M matter at scale? M controls the per-node degree in the HNSW graph. As N grows, the graph diameter increases — greedy search needs more connections per node to navigate reliably to the true nearest neighbour. ef_search controls search breadth at query time; M controls graph connectivity at build time. These are orthogonal parameters.

---

vs Embedded Competitors (usearch, hnswlib)

Head-to-head on cosine workloads, single-core, release + x86-64-v3, all builds single-threaded for fairness:

AG News 45K × 1024:

System	recall@10	QPS	Mem
MonaVec BF 4-bit	0.960	137	27 MB
MonaVec HNSW 4-bit	0.954	1,244	249 MB
usearch HNSW i8 (8-bit)	0.928	5,726	55 MB
usearch HNSW f32	0.987	2,388	202 MB
hnswlib HNSW f32	0.995	2,194	191 MB
FAISS-IVF f32 (nprobe=10)	0.936	2,597	40 MB
sqlite-vec (exact brute f32)	1.000	27	140 MB

glove-100 1.18M × 100:

System	recall@10	QPS
MonaVec BF 4-bit	0.865	42
MonaVec HNSW 4-bit	0.801	352
usearch HNSW f32	0.756	2,598
hnswlib HNSW f32	0.827	5,184
sqlite-vec (exact brute f32)	—	impractical at 1.18M

Note on sqlite-vec: It performs exact brute-force (recall 1.000), so it's the accuracy ceiling — but it doesn't quantize, so it doesn't scale: at 45K it already runs at 27 QPS, and at 1.18M exact brute-force is impractical (skipped). MonaVec's 4-bit BruteForce scans the same 1.18M corpus at 42 QPS — the quantization-as-scaling advantage that separates us from the closest "SQLite of vector search" positioning.

Honest read:

• Recall: MonaVec leads where it counts. On AG News we beat usearch-i8 (0.960 vs 0.928) at half the bytes (4-bit vs 8-bit) and half the memory (27 vs 55 MB), and our HNSW (0.954) tops FAISS-IVF (0.936). On glove-100 our BruteForce (0.865) tops every graph index including hnswlib f32 (0.827), and our HNSW (0.801) beats usearch's HNSW (0.756) — the auto-M=64 payoff at 1M scale. Uncompressed float32 graphs (usearch-f32 0.987, hnswlib 0.995) sit above us, as expected — they store 8× the data.
• Throughput: we trail 2–14×. usearch (simsimd), hnswlib, and current FAISS releases are mature, hand-tuned SIMD C++; our scoring pays a 4-bit unpack cost and our build is single-threaded by design (see Determinism). This is an optimization gap, not an architectural ceiling. (FAISS numbers use the current release, which is substantially faster than older builds — we report present performance, not a favorable snapshot.)
• Memory: BruteForce is best-in-class (zero-copy ingestion). HNSW still carries an FP32 build buffer + graph overhead — a known target for compression.

Takeaway: MonaVec is the recall-and-determinism choice, not the raw-speed choice. For semantic retrieval where correctness and reproducibility matter more than peak QPS, it leads; for pure throughput on a single warm server, mature HNSW libraries are faster.

Where we stand, and where we're going. The position is asymmetric by kind: our advantages — highest recall at 4-bit, best-in-class BruteForce memory, unique portable determinism — are mathematical (RHDH + Lloyd-Max + auto-M + asymmetric scoring) and durable. Our deficit — 2–15× QPS — is engineering: the gap from compiler-vectorized Rust to a decade of hand-tuned SIMD C++. Engineering gaps close; mathematical advantages persist. We are closing it without spending what defines us — every throughput lever preserves the float32 query and determinism:

• Scoring kernel — 4 independent accumulators to hide FMA latency (done, −37% per-vector scoring in profiling); wider 16-dim loads + prefetch next.
• HNSW memory — uint40 neighbours + streamed FP32 build vectors (targeting 249 → ~60 MB) without changing graph topology.
• AVX-512 VNNI on server targets via the existing runtime dispatch.

We deliberately decline int8 query quantization — the fastest gap-closer (it's how usearch-i8 hits 5,700 QPS) — because symmetric quantization would trade away the recall lead and determinism that are the whole point. The goal isn't "become a faster HNSW library"; it's keep the recall + determinism crown while making throughput a non-issue for edge/offline targets.

---

SIMD speedup: 4.1–4.75× vs scalar (AVX2+FMA). AVX-512 available on Intel Ice Lake+ / AMD Zen 4+.

Memory:

Vectors	Dim	MonaVec 4-bit	FP32
100K	768	36 MB	290 MB
1M	1536	768 MB	6.1 GB

---

Index Types

Type	Build	Search	Best For
BruteForce	O(1) per insert	O(n) scan	n < 500K, edge, deterministic
IvfFlat	k-means (one-time)	O(n/nlist × nprobe)	High QPS, n > 100K
Hnsw	Incremental O(log n)	O(log n)	Lowest latency, n > 100K

• Edge / offline / < 100K vectors → BruteForce. Zero build time, no parameters.
• High throughput → IvfFlat. Start with nlist = √n, tune nprobe.
• Lowest latency → Hnsw. Use M=32 for N < 1M, M=64 for N ≥ 1M. Minimum M=32 for 4-bit indexes.

4-bit HNSW note: Quantization noise (~0.01–0.02) exceeds typical inter-neighbor score gaps (~0.001–0.003) on normalized embeddings. M=32 is the reliable minimum; M=16 is sufficient for float32 but not for 4-bit. MonaVec builds the graph in FP32 and only uses 4-bit for storage and query scoring — this preserves graph topology.

---

How It Works

Quantization Pipeline

Cosine / Dot:
  input f32[dim]
    → normalize (unit length)       ← Cosine only
    → RHDH rotate                   (ChaCha20-seeded Walsh-Hadamard, O(d log d))
    → scalar quantize               (Lloyd-Max centroids for N(0,1), 16 levels at 4-bit)
    → nibble pack                   (4-bit: 2 values/byte → 8× compression)

L2 (after fit):
  input f32[dim]
    → standardize per-dim           (x - mean) / std  ← fit() computes this once
    → RHDH rotate
    → scalar quantize
    → nibble pack

After RHDH, coordinates approximate N(0,1). Lloyd-Max centroids minimize MSE for this distribution.

For Cosine, unit normalization ensures N(0,1) after rotation — no extra step needed. For L2, vector magnitude carries distance information and must not be removed; per-dimension standardization shifts and scales each dimension independently, preserving inter-vector distances while restoring the N(0,1) assumption for the quantizer.

# L2 indexes: call fit() before add()
index = MonaVec(dim=784, metric="l2", bit_width=4)
index.fit(sample_vecs)   # one pass over a representative sample
index.add(ids, all_vecs)
index.search(query, k=10)

SIMD Dispatch

is AVX-512F+BW?  → avx512 kernel   (16 dims/iter, single-register 16-centroid lookup)
is AVX2+FMA?     → avx2 kernel     (4.1–4.75× vs scalar)
is NEON?         → neon kernel     (aarch64 / Apple Silicon)
fallback         → scalar          (always-correct reference)

Optional Build Tuning: x86-64-v3 (+43%)

The published PyPI/crates.io builds use the generic x86-64 baseline so they run on every x86 CPU — a wheel that assumes a newer instruction set would crash with SIGILL on older hardware (the manylinux tag can't encode micro-architecture, so pip can't protect against it).

If your deployment CPUs are Haswell-2013 or newer, you can opt into the x86-64-v3 baseline (AVX2 + FMA + BMI2) when building from source. This lets the compiler auto-vectorize all code paths — not just the runtime-dispatched nibble kernel — for a measured +43% QPS on AG News BruteForce (96 → 137), matching target-cpu=native while staying portable across all post-2013 x86:

# Build from source with the v3 baseline
RUSTFLAGS="-C target-cpu=x86-64-v3" pip install --no-binary :all: monavec
# or for the Rust crate:
RUSTFLAGS="-C target-cpu=x86-64-v3" cargo build --release

Runtime SIMD dispatch (AVX-512 / AVX2 / NEON) and determinism are unaffected either way — the generic and v3 builds each use a fixed baseline, so results stay byte-identical across machines using the same build. All benchmarks in this README were measured with the v3 opt-in.

Zero-Copy Ingestion

When you pass a C-contiguous float32 NumPy matrix to add(), MonaVec reads it as a flat slice straight into the encoder — no intermediate Vec<Vec<f32>> allocation. This cut peak memory by 87% on AG News BruteForce (210 → 27 MB) and 43% for HNSW (435 → 249 MB), with byte-identical results. Lists-of-lists and non-contiguous arrays transparently fall back to the copy path.

Sequential Build by Design (Determinism)

MonaVec builds indexes sequentially, single-threaded — a deliberate choice, not a missing optimization. Parallel graph construction (as in hnswlib/usearch) makes insertion order non-deterministic, which makes the resulting graph non-deterministic: the same vectors produce a different index on each run. MonaVec guarantees the opposite — the same input and seed always produce a byte-identical .mvec on any machine. For medical devices, offline agents, and reproducible research, this determinism is worth more than a faster build. We are aware build time trails parallel implementations; closing that gap without sacrificing determinism is on the roadmap (e.g. a hand-written AVX2 build-distance kernel).

Pre-filtering

The allowlist is resolved with a HashSet O(1) lookup before scoring begins. Post-filtering degrades recall when the candidate pool is small — pre-filtering avoids this entirely.

.mvec File Format (v6)

[MAGIC 4B]       b"MVEC"
[VERSION 4B]     u32 (current: 6)
[DIM 4B]         u32
[METRIC 1B]      u8  — 0=Cosine, 1=Dot, 2=L2
[BIT_WIDTH 1B]   u8  — 2 or 4
[INDEX_TYPE 1B]  u8  — 0=BruteForce, 1=IvfFlat, 2=Hnsw
[PAD 1B]         reserved
[COUNT 8B]       u64
[SEED 8B]        u64 — ChaCha20 seed (determinism)
[N4_DIMS 4B]     u32 — dims at 4-bit in padded space
[PARAMS 8B]      index-specific tuning params
[HAS_STD 1B]     u8  — 1 if per-dim standardization params follow (v6+, L2 only)
[PAD 1B]         reserved
[STD_MEAN ...]   f32[dim] — per-dim mean  (only when HAS_STD=1)
[STD_INV_STD ...] f32[dim] — per-dim 1/std (only when HAS_STD=1)
[VECTORS ...]    packed quantized data
[IDS ...]        u64 IDs
[NORMS ...]      f32 per-vector norms
[INDEX_DATA ...] IvfFlat centroid+list data or HNSW graph

The seed is embedded so that load → search produces identical results to the original index, on any machine, always.

Backward compatibility: v1–v5 BruteForce, IvfFlat, and HNSW files load correctly.

⚠️ Breaking change in v5: HNSW indexes built with v4 are incompatible (neighbor count type changed u8→u16). Rebuild any v4 HNSW collections.

---

Supported Metrics

Metric	Notes
cosine	Normalize → dot product. Inputs normalized internally. No extra steps.
dot	Raw dot product. Vector magnitude contributes to score.
l2	Euclidean distance. Returns negated squared L2 (higher = closer). Call fit() before add() for raw-magnitude vectors (pixels, SIFT, etc.).

L2 and quantization quality: Lloyd-Max tables assume N(0,1) input. Cosine achieves this via unit normalization. For L2, fit() computes a global (scalar) mean and std across all sample values and applies (x − mean) / std uniformly — restoring the N(0,1) assumption while preserving relative L2 distances. On embedding vectors already close to unit norm, fit() has negligible effect and can be skipped.

On fashion-mnist-784 (raw pixels), fit() improves Recall@10 from 0.41 → 0.62. The remaining gap to trained PQ methods (~0.85) is the inherent cost of MonaVec's zero-training design. For semantic embeddings (BGE, OpenAI), recall reaches 0.95+.

---

Admin UI & CLI

MonaVec ships with a FastAPI web UI and Typer CLI for managing collections without writing code.

Collections dashboard	Collection detail — quantization & sparse index stats
Vectors browser — payload + dense/sparse tags	Embedding visualizer — 2D projection

# Start the server (port 7333)
monavec serve

# Collection management
monavec list
monavec info my_collection
monavec backup my_collection
monavec restore my_collection.zip

# Search from the terminal
monavec search my_collection --k 10
monavec search my_collection --mode sparse --text "machine learning"
monavec search my_collection --mode hybrid --text "RAG" --alpha 0.5
monavec search my_collection --filter '{"category": "rag"}' --k 5

Authentication: No token → __public__ namespace. Bearer token → per-user namespace (standalone or identity service — see Auth Integration).

The UI supports live ingestion monitoring — browse and search a collection while vectors are still being added.

---

Auth Integration

MonaVec supports two auth modes — no code changes required, just environment variables.

Standalone mode (default)

No external service needed. The Bearer token is used directly as the namespace key.

# Personal use — each token gets its own isolated collection space
export MONAVEC_TOKEN=alice
monavec serve
# collections stored under collections/alice/

No token → __public__ namespace (open access).

Python client:

# Explicit token
col = MonaVecClient.create("products", dim=768, host="http://localhost:7333", token="alice")
col = MonaVecClient.load("products",           host="http://localhost:7333", token="alice")

# Via env var — convenient for CI/CD and Docker
# export MONAVEC_TOKEN=alice
col = MonaVecClient.load("products", host="http://localhost:7333")

# No token → __public__ namespace
col = MonaVecClient.load("products", host="http://localhost:7333")

Token priority: token= parameter → MONAVEC_TOKEN env var → __public__.

Identity service mode

Set IDENTITY_URL to any service that verifies tokens. MonaVec calls:

GET {IDENTITY_URL}/api/v1/identity/verify
Authorization: Bearer <token>

Success response (HTTP 200):

{"success": true, "data": {"user_id": "alice"}}

data.user_id becomes the namespace key — each unique value gets an isolated collection space.

Failure: return any non-200 status, or {"success": false, ...}. MonaVec responds with HTTP 401.

Resilience: responses are cached for 30 seconds. If the identity service becomes unreachable, the last valid response is served from cache.

# .env
IDENTITY_URL=http://localhost:7002

This is the OAuth2 token introspection pattern — any system that can expose this single endpoint works:

System	How
Keycloak	Point to the Keycloak introspection endpoint with a thin adapter
Auth0	/userinfo endpoint wrapped in a small proxy
JWT (custom)	Decode and verify the token, return user_id from claims
LDAP / Active Directory	Bind with token as credential, return user_id on success
API key table	DB lookup: token → user_id, one endpoint

A minimal adapter in any language:

# FastAPI example — sits in front of your auth system
@app.get("/api/v1/identity/verify")
async def verify(authorization: str = Header(...)):
    token = authorization.removeprefix("Bearer ")
    user_id = your_auth_system.verify(token)   # your logic here
    if not user_id:
        raise HTTPException(401)
    return {"success": True, "data": {"user_id": user_id}}

---

Demo Notebooks

Three Jupyter notebooks in demo/:

Notebook	What It Shows
demo_local.ipynb	Embedded mode — no server, pure Python
demo_server.ipynb	REST API via httpx
demo_client.ipynb	MonaVecClient high-level client

---

Competitive Position

Capability	MonaVec	FAISS	Qdrant	Vespa
Embedded (no server)	✅	✅	❌	❌
Edge / offline	✅	❌	❌	❌
Zero training	✅	❌	❌	❌
BruteForce + IvfFlat + HNSW	✅	✅	HNSW only	❌
Built-in quantization	✅ Lloyd-Max	✅ PQ	✅ SQ/PQ	partial
Hybrid dense+sparse	✅ BM25 + SPLADE	❌	✅	✅
Admin UI	✅	❌	✅	✅

MonaVec's position: SQLite of vector search — embedded, zero-config, offline-first.

---

Roadmap

v1.x core is stable. Three capabilities are planned for future versions:

Feature	What It Unlocks
WASM target (monavec-wasm)	Vector search in the browser, Cloudflare Workers, and Deno Deploy — no server, no round-trip. Pure Rust with zero C dependencies makes this a natural next step.
ColPali / multi-vector late interaction	One document → N vectors. MaxSim late interaction scoring. Semantic search over images and documents without chunking — enters Vespa territory, still embedded.
Per-dimension bit permutation	Variance-sorted bit assignment: high-variance dimensions get 4-bit, low-variance get 2-bit. Target: recall@10 ≥ 0.95 at avg 3.0 bits/dim. Format v6.

---

Known Limitations

• Dot quantization quality — tables optimized for N(0,1); recall may be slightly lower on highly unnormalized dot-product vectors.
• L2 without fit() — on raw-magnitude data (pixels, SIFT), skipping fit() degrades recall. Embedding vectors (BGE, OpenAI) are already near unit norm — fit() optional there.
• Sparse search — BM25 and SPLADE sparse embeddings supported. Hybrid dense+sparse via RRF.
• Payload filter index is in-memory — rebuilt from disk on first query after restart.
• 2-bit SIMD — scalar fallback; AVX2/NEON for 2-bit planned post-v1.0.
• IvfFlat allowlist — post-filter on probed cells; full pre-filter planned.

---

Contributing

Issues and PRs are welcome. A few pointers:

• The scalar implementation in crates/monavec-core/src/simd/mod.rs is the correctness reference — all SIMD paths are tested against it.
• New SIMD kernels: assert_approx_eq!(simd_result, scalar_result, epsilon=1e-4).
• Quantization changes: recall@10 ≥ 0.90 on synthetic Gaussian data is the acceptance bar.
• No async runtime, no HTTP, no distributed systems code in core.

---

License

Licensed under either of MIT or Apache-2.0 at your option.

---

Monachus monachus — Akdeniz'in en nadir memelisi. Yaklaşık 700 birey.
The rarest marine mammal of the Mediterranean. Roughly 700 individuals left.

mona-hq — Embedded AI infrastructure.