m2vdb 1.0.0

M2VDB - Understanding Vector Search Through Real Implementations

This project is simply me trying to understand vector search and databases from first principles, while having fun building something end-to-end that feels like a real vector DB. I’ve worked as an applied scientist on AI systems with retrieval and yet, I never really understood how vector databases actually work. Until now :)

✨ Features

🗺️ Roadmap

• More Indexe: Implement HNSW (Python first, Rust when I'm board).
• Comparative Benchmarks: Add FAISS baselines to compare my implementations.
• Experiments: Hyperparameter sweeps for PQ (and others) with visualization/graphs.
• Configuration: Better config management for running benchmark sweeps.
• Memory Benchmarking: Improve memory measurement to track non-Python indexes.
• MCP Server: Model Context Protocol integration (because why not?).

⚡️ Quick Start

Installation

Option 1: From PyPI (Recommended)

pip install m2vdb
# or with uv
uv pip install m2vdb

Option 2: From Source

git clone https://github.com/mmilunovic/m2vdb.git
cd m2vdb
uv sync

Optional: Enable Rust Indexes

For maximum performance, you can build optional Rust extensions:

# Install Rust if you don't have it
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Build Rust indexes
cd rust
maturin develop --release
cd ..

Start the Server

Using Docker

docker-compose up -d

Using CLI Command

# Basic usage
m2vdb-server

# Custom port
m2vdb-server --port 8080

# With persistent storage (when implemented)
m2vdb-server --data-dir /path/to/data

# Development mode with auto-reload
m2vdb-server --reload

💡 Tip: Once the server is running, visit http://localhost:8000/docs for the interactive API documentation (Swagger UI) to explore endpoints and test requests directly from your browser.

Use the Client

from m2vdb import M2VDBClient

# 1. Connect
client = M2VDBClient(api_key="sk-test-user1", host="http://localhost:8000")

# 2. Create Index
index = client.create_index(
    name="demo", 
    dimension=3, 
    metric="cosine",
    index_type="brute_force"  # Options: "brute_force", "pq", "ivf", "rust_brute_force" (if built)
)

# 3. Insert Data
index.upsert(
    vectors=[
        {"id": "A", "vector": [1.0, 0.0, 0.0], "metadata": {"label": "Red"}},
        {"id": "B", "vector": [0.0, 1.0, 0.0], "metadata": {"label": "Green"}},
    ]
)

# 4. Search
results = index.query(
    vector=[0.9, 0.1, 0.0],
    top_k=1
)
print(results) # Matches "A" (Red)

Using Rust Indexes (Optional)

If you've built the Rust extensions, you can use them for significantly better performance:

from m2vdb import Collection, HAS_RUST

# Check if Rust is available
print(f"Rust indexes available: {HAS_RUST}")

# Use Rust brute force index (5-10x faster than Python)
db = Collection(
    dimension=128,
    metric="euclidean",
    index_type="rust_brute_force"  # Requires Rust extensions
)

# Or use it via the client
index = client.create_index(
    name="fast-demo",
    dimension=128,
    metric="euclidean", 
    index_type="rust_brute_force"
)

Performance comparison (1M vectors, 128D):

• Python BruteForce: ~5 QPS
• Rust BruteForce: ~25 QPS (5x faster!)

📊 Benchmarks

All results below were generated on a MacBook Air M4, 16GB RAM, with:

• 1,000,000 base vectors
• 1,000 queries
• k = 10

SIFT1M (1M vectors, 128D)

FASTTEXT (sampled 1M vectors, 300D)

To reproduce results just run.

uv run python benchmarks/run_benchmarks.py

📜 License

MIT. If you actually use it I'll be flattered 🥹