SimpleVecDB

A local-first, embedded vector database backed by SQLite and usearch.

SimpleVecDB pairs Chroma-like ergonomics with a file-based store — a SQLite database for metadata and text alongside a usearch HNSW index per collection. It provides high-performance vector search, quantization, and hybrid retrieval with no separate services to run. It fits local RAG pipelines, offline agents, and any application that needs production-grade vector search without the operational overhead of a hosted database.

Why SimpleVecDB?

• Zero Infrastructure — Local files on disk: a SQLite database plus a usearch index. No Docker, no Redis, no external services.
• High Performance — usearch HNSW indexing with adaptive search: brute-force under 10k vectors (perfect recall), HNSW above that.
• Portable — Runs anywhere SQLite runs: Linux, macOS, Windows, and WASM.
• Async Support — A complete async/await surface with optional executor injection for thread-safe ONNX/usearch sharing.
• Integrations Included — Optional FastAPI embeddings server and LangChain/LlamaIndex adapters via the [integrations] extra.
• Production Ready — Hybrid search (BM25 + vector), metadata filtering, multi-collection support, and automatic hardware acceleration.

When to Choose SimpleVecDB

Prerequisites

System Requirements:

• Python 3.10+
• SQLite 3.35+ with FTS5 support (included in Python 3.8+ standard library)
• 50MB+ disk space for core library, 500MB+ with [server] extras

Optional for GPU Acceleration:

• CUDA 11.8+ for NVIDIA GPUs
• Metal Performance Shaders (MPS) for Apple Silicon

Note: If using custom-compiled SQLite, ensure -DSQLITE_ENABLE_FTS5 is enabled for full-text search support.

Installation

# Standard installation (includes clustering, encryption)
pip install simplevecdb

# With LangChain & LlamaIndex integrations
pip install "simplevecdb[integrations]"

# With local embeddings server (adds 500MB+ models)
pip install "simplevecdb[server]"

What's included by default:

• Vector search with HNSW indexing
• Clustering (K-means, MiniBatch K-means, HDBSCAN)
• Encryption (SQLCipher AES-256)
• Async support

Verify Installation:

python -c "import simplevecdb; print(simplevecdb.__version__)"

Quickstart

SimpleVecDB is just a storage and search layer — it doesn't ship an LLM and won't generate embeddings for you. Bring whichever embedding source you already use; three common ones below.

Option 1: OpenAI embeddings

from simplevecdb import VectorDB
from openai import OpenAI

client = OpenAI()
db = VectorDB("notes.db")
notes = db.collection("personal")

def embed(text: str) -> list[float]:
    return (
        client.embeddings
        .create(model="text-embedding-3-small", input=text)
        .data[0].embedding
    )

entries = [
    ("Cherry MX silent reds bottom out around 45g — quieter than browns", "keyboards"),
    ("Sourdough hydration sweet spot is ~75% with this flour",            "baking"),
    ("EXPLAIN ANALYZE showed seq scan; ANALYZE on the table fixed it",    "work"),
    ("Passport renewal took 3 weeks, not the advertised 6–8",             "admin"),
]

notes.add_texts(
    texts=[t for t, _ in entries],
    embeddings=[embed(t) for t, _ in entries],
    metadatas=[{"tag": tag} for _, tag in entries],
)

hits = notes.similarity_search(embed("how loud are silent reds"), k=2)
for doc, score in hits:
    print(f"{score:.3f}  {doc.page_content}")

work = notes.similarity_search(
    embed("query plan slow"),
    k=5,
    filter={"tag": "work"},
)

Option 2: Fully local (no network, no API key)

pip install "simplevecdb[server]"

from simplevecdb import VectorDB
from simplevecdb.embeddings.models import embed_texts

db = VectorDB("notes.db")
notes = db.collection("personal")

texts = [
    "Cherry MX silent reds bottom out around 45g",
    "Sourdough hydration sweet spot is ~75% with this flour",
    "EXPLAIN ANALYZE showed seq scan; ANALYZE on the table fixed it",
]
notes.add_texts(texts=texts, embeddings=embed_texts(texts))

vec = notes.similarity_search(embed_texts(["quieter switches"])[0], k=2)
mixed = notes.hybrid_search("postgres slow query", k=3)

If you'd rather hit an HTTP endpoint than import the embedding models directly, the bundled server speaks the same shape as OpenAI's embeddings API:

simplevecdb-server --port 8000                # default model, auto warm-up
simplevecdb-server --host 0.0.0.0 --port 9000
simplevecdb-server --no-warmup                # skip the model preload
simplevecdb-server --help

Server tuning (model registry, rate limits, API keys, CORS, CUDA) lives in the Setup Guide.

Option 3: LangChain or LlamaIndex

Already using one of the major RAG frameworks? Use SimpleVecDB as the vector store:

pip install "simplevecdb[integrations]"

from simplevecdb.integrations.langchain import SimpleVecDBVectorStore
from langchain_openai import OpenAIEmbeddings

store = SimpleVecDBVectorStore(
    db_path="notes.db",
    embedding=OpenAIEmbeddings(model="text-embedding-3-small"),
)

store.add_texts([
    "Cherry MX silent reds bottom out around 45g",
    "EXPLAIN ANALYZE showed seq scan; ANALYZE on the table fixed it",
])
store.similarity_search("quieter switches", k=1)
store.hybrid_search("postgres performance", k=3)

LlamaIndex is the same shape:

from simplevecdb.integrations.llamaindex import SimpleVecDBLlamaStore
from llama_index.embeddings.openai import OpenAIEmbedding

store = SimpleVecDBLlamaStore(
    db_path="notes.db",
    embedding=OpenAIEmbedding(model="text-embedding-3-small"),
)

End-to-end notebooks (including a fully local Ollama RAG) live in the examples gallery.

Feature Highlights

A few of the things SimpleVecDB does well — see docs/Features.md for the full list.

• Vector + keyword + hybrid search — cosine / L2 similarity, BM25 via SQLite FTS5, and Reciprocal Rank Fusion in one collection.
• Adaptive HNSW — brute-force for <10k vectors (perfect recall), usearch HNSW above that. Override per query with exact=True/False.
• Quantization — FLOAT32, FLOAT16, INT8, BIT for 1×–32× compression.
• Multi-collection + cross-collection search — isolated namespaces in one database, with merged ranked search across them.
• Mongo-style filters — $eq $ne $gt $gte $lt $lte $in $nin $exists $between on metadata, edges, and events.
• Memory primitives (v2.6.1) — pending-vector buffer with atomic flush, weighted directed edges, append-only event feed, TTL with delete/callback sweep, and a threshold-driven rebuild scheduler.
• Atomic counters & transactions (v2.6.1) — increment_metadata for JSON deltas in one statement; SAVEPOINT-backed db.transaction() / collection.tx() rolling catalog and vector writes back on error, with reserve_ids() for ids you need before the rows exist.
• Async, encryption, clustering, hierarchies — full async surface (with executor injection), SQLCipher AES-256, K-means / MiniBatch K-means / HDBSCAN, parent/child relationships.
• Framework integrations — drop-in LangChain and LlamaIndex adapters via the [integrations] extra; optional FastAPI embeddings server via [server].

For full method-level coverage, see the Features doc or the API reference.

Performance Benchmarks

10,000 vectors, 384 dimensions, k=10 search — Full benchmarks →

Key highlights:

• 3-34x faster than brute-force for collections >10k vectors
• Adaptive search: perfect recall for small collections, HNSW for large
• FLOAT16 recommended: best balance of speed, memory, and precision

Documentation

• Features — Every capability, grouped by area
• Setup Guide — Environment variables, server configuration, authentication
• API Reference — Complete class/method documentation with type signatures
• Benchmarks — Quantization strategies, batch sizes, hardware optimization
• Integration Examples — RAG notebooks, Ollama workflows, production patterns
• Contributing Guide — Development setup, testing, PR guidelines

Troubleshooting

Import Error: sqlite3.OperationalError: no such module: fts5

# Your Python's SQLite was compiled without FTS5
# Solution: Install Python from python.org (includes FTS5) or compile SQLite with:
# -DSQLITE_ENABLE_FTS5

Dimension Mismatch Error

# Ensure all vectors in a collection have identical dimensions
collection = db.collection("docs", dim=384)  # Explicit dimension

CUDA Not Detected (GPU Available)

# Verify CUDA installation
python -c "import torch; print(torch.cuda.is_available())"

# Reinstall PyTorch with CUDA support
pip install torch --index-url https://download.pytorch.org/whl/cu118

Slow Queries on Large Datasets

• Enable quantization: collection = db.collection("docs", quantization=Quantization.INT8)
• For >10k vectors, HNSW is automatic; tune with rebuild_index(connectivity=32)
• Use exact=False to force HNSW even on smaller collections
• Use metadata filtering to reduce search space

Roadmap

What's on the near-term radar:

• Incremental clustering (online learning)
• Cluster visualization exports

For shipped capabilities, see docs/Features.md and the release-by-release Changelog. Vote on these or propose new ideas in GitHub Discussions.

Contributing

Contributions are welcome — bug fixes, documentation improvements, and new feature proposals alike:

1. Read CONTRIBUTING.md for development setup
2. Check existing Issues and Discussions
3. Open a PR with clear description and tests

Community & Support

Get Help:

• GitHub Discussions — Q&A and feature requests
• GitHub Issues — Bug reports

Stay Updated:

• GitHub Releases — Changelog and updates
• Examples Gallery — Community-contributed notebooks

Support the Project

• Sponsor on Ko-fi — one-time donations
• Star the repository — helps with visibility
• Report issues — bug reports and feedback
• Contribute — development setup and guidelines

License

MIT License — Free for personal and commercial use.