surgedb 1.0.0a5

A high-performance, lightweight vector database

SurgeDB Python

SurgeDB is a high-performance, embedded vector database for Python. It runs entirely locally—no Docker containers, no external APIs, and no complex setup required.

Built in Rust and powered by SIMD-accelerated HNSW indices, SurgeDB offers millisecond-latency vector search with a minimal memory footprint.

---

Key Features

• 🚀 Blazing Fast: Hand-tuned AVX-512 and NEON kernels for maximum throughput.
• 🧠 Memory Efficient: Built-in SQ8 (4x) and Binary (32x) quantization. Index millions of vectors on a laptop.
• 📦 Embedded: Runs in-process. Just pip install and go.
• 💾 Persistent: ACID-compliant storage with Write-Ahead Logs (WAL) and crash-safe snapshots.
• 🔍 Rich Filtering: Filter search results by metadata (exact match, comparison, logical operators).

---

Installation

pip install surgedb

---

Quick Start

from surgedb import SurgeClient, SurgeConfig, DistanceMetric, Quantization

# 1. Initialize a persistent database
config = SurgeConfig(
    dimensions=384,                     # e.g., for all-MiniLM-L6-v2
    distance_metric=DistanceMetric.COSINE,
    quantization=Quantization.SQ8,      # 4x compression
    persistent=True,
    data_path="./my_vector_db"
)
db = SurgeClient.open("./my_vector_db", config)

# 2. Insert data (ID, Vector, Metadata)
db.insert(
    "doc_1", 
    [0.1, 0.2, 0.3, ...],               # 384-dim list or numpy array
    '{"title": "How to train your dragon", "tag": "movie"}'
)

# 3. Search with metadata filtering
results = db.search_with_filter(
    query=[0.1, 0.2, 0.3, ...], 
    k=5, 
    filter={'Exact': {'field': 'tag', 'value': 'movie'}}
)

for result in results:
    print(f"ID: {result.id}, Score: {result.score:.4f}")

---

Advanced Usage

Batch Operations

For maximum write throughput, use upsert_batch.

vectors = []
for i in range(1000):
    vectors.append({
        "id": f"vec_{i}",
        "vector": [0.1] * 384,
        "metadata": {"index": i}
    })

db.upsert_batch(vectors)

Metadata Filtering

SurgeDB supports a structured query language for filtering.

from surgedb import SearchFilter

# Find movies released after 2020 OR in the "Sci-Fi" genre
filter_query = SearchFilter.Or([
    SearchFilter.Comparison(field="year", operator="gt", value=2020),
    SearchFilter.Exact(field="genre", value="Sci-Fi")
])

results = db.search_with_filter(query_vec, 10, filter_query)

---

Performance

SurgeDB is designed to outperform pure-Python solutions and compete with heavy C++ vector stores, while remaining lightweight.

Metric	Performance
Search Latency	< 1ms (1M vectors, SQ8)
Indexing Speed	~20k vectors/sec
Memory (1M vectors)	~120MB (SQ8) vs ~4GB (Float32)
Cold Start	< 50ms

---

Development

If you want to contribute to the bindings or build from source:

Prerequisites

• Rust toolchain (stable)
• Python 3.7+
• maturin build tool

Building from Source

# Install maturin
pip install maturin

# Build and install locally
maturin develop --release -m crates/surgedb-bindings/Cargo.toml

Generating Bindings (UniFFI)

The bindings are automatically generated using UniFFI.

cd crates/surgedb-bindings
make generate-python