🚀 Why NanoVector?

Modern AI agents and local LLM pipelines are plagued by vector database bloat:

• ChromaDB, Pinecone clients, and FAISS pull hundreds of megabytes of dependencies (torch, onnxruntime, pydantic, fastapi, duckdb).
• Cold Start Penalty: Importing Chroma takes 1.5 to 2.5 seconds, crippling CLI tools, serverless workers (AWS Lambda), and autonomous agent loops.
• The Small-to-Medium Vector Trap: Over 95% of AI agents store between 50 and 50,000 vectors (conversation turns, tool execution history, episodic facts). At this scale, graph traversal (HNSW) incurs heavy pointer indirection, high memory overhead, and non-deterministic recall.

NanoVector solves this by delivering exact, sub-millisecond, brute-force SIMD search directly in CPU cache with zero external dependencies.

⚡ Installation

Install the zero-dependency pre-compiled binary wheel in under 1 second:

pip install nanovector

🏁 Quickstart

import nanovector
import numpy as np

# 1. Initialize an index (dim=384 for all-MiniLM-L6-v2, 768 for BERT, 1536 for OpenAI)
index = nanovector.Index(dim=384, metric="cosine")

# 2. Add single embeddings with optional metadata strings
vec = np.random.randn(384).astype(np.float32)
index.add("doc_1", vec, metadata='{"author": "eminsk", "tag": "ai"}')

# 3. Batch addition (Zero-Copy directly from 2D NumPy array)
batch_vecs = np.random.randn(5000, 384).astype(np.float32)
batch_ids = [f"turn_{i}" for i in range(5000)]
batch_metas = [f'{{"turn_id": {i}, "role": "agent"}}' for i in range(5000)]
index.add_batch(batch_ids, batch_vecs, metadatas=batch_metas)

# 4. Search top-k nearest neighbors (returns in ~0.15 ms)
query = np.random.randn(384).astype(np.float32)
results = index.search(query, top_k=5)

for r in results:
    print(f"[{r.id}] Score: {r.score:.4f} | Metadata: {r.metadata}")

# 5. Single-file instant persistence (.nvec)
index.save("agent_memory.nvec")

# 6. Instant reload from disk
loaded_index = nanovector.load("agent_memory.nvec")
print(f"Reloaded {len(loaded_index)} vectors in {loaded_index.dim}D")

AI Agent Episodic Memory Pattern

Give your LLM agents lightning-fast, persistent long-term memory:

import nanovector
import numpy as np

class AgentEpisodicMemory:
    def __init__(self, filepath="agent_brain.nvec", dim=384):
        self.filepath = filepath
        try:
            self.index = nanovector.load(filepath)
        except Exception:
            self.index = nanovector.Index(dim=dim, metric="cosine")

    def remember(self, fact_id: str, embedding: np.ndarray, fact_text: str):
        self.index.add(fact_id, embedding, metadata=fact_text)
        self.index.save(self.filepath)

    def recall(self, query_embedding: np.ndarray, top_k=3):
        return self.index.search(query_embedding, top_k=top_k)

# Usage in Agent Loop
memory = AgentEpisodicMemory(filepath="agent_brain.nvec")

# Store facts if brain is empty
if len(memory.index) == 0:
    memory.remember("mem_1", np.random.randn(384).astype(np.float32), "User prefers Python, C, and FASM.")
    memory.remember("mem_2", np.random.randn(384).astype(np.float32), "NanoVector achieves sub-millisecond search.")
    memory.remember("mem_3", np.random.randn(384).astype(np.float32), "Episodic memory saves state in single .nvec file.")

query_vec = np.random.randn(384).astype(np.float32)
recalled_facts = memory.recall(query_vec, top_k=3)

for match in recalled_facts:
    print(f"Score: {match.score:.4f} -> Memory: {match.metadata}")

🚀 Interactive Google Colab Demo

Run NanoVector interactively in your browser with zero local setup:

The Interactive Colab Notebook demonstrates:

• Zero-Setup Installation & Hardware SIMD Detection: Compiles native C/AVX2 on Colab CPU in seconds.
• 10-line Cosine Similarity Search: Indexing and querying embeddings with JSON metadata.
• Real-World AI Agent Episodic Memory: Recalling instructions and preferences using sentence-transformers embeddings (all-MiniLM-L6-v2).
• Single-File .nvec Brain Persistence: Instant binary save and zero-overhead reload.
• Live 50,000-Vector Benchmark: Measuring ingestion throughput (1M+ vectors/sec) and search latency (~0.1 ms) directly on Colab VM hardware.

📊 Benchmarks

Real-world benchmarks measured on Intel/AMD x86_64 CPU (AVX2+FMA) using standard 384-dimensional sentence embeddings (all-MiniLM-L6-v2) against NumPy 2.x / OpenBLAS:

Single-Threaded Exact Search Latency

High-Throughput Batch Ingestion & Persistence

• Ingestion Throughput: 1,414,447 vectors/sec (20,000 512D vectors ingested in 14.14 ms via Zero-Copy Buffer Protocol).
• Multi-Threaded Concurrency (8 threads): 14,300 QPS (400 concurrent queries executed in 27.97 ms with zero lock contention).
• Persistence Serialization: Save 2,000 vectors in 1.71 ms, load in 3.92 ms (single binary .nvec file).

🏛️ Architecture & Acceleration

NanoVector is written in standard C99 with a multi-tiered hardware acceleration pipeline:

                  ┌───────────────────────────────┐
                  │       Python C-API            │
                  │  (Buffer Protocol / No-GIL)   │
                  └───────────────┬───────────────┘
                                  │
                  ┌───────────────▼───────────────┐
                  │      NanoVector C99 Core      │
                  │   Top-K In-Place Heap $O(N\log K)$  │
                  └───────────────┬───────────────┘
                                  │
         ┌────────────────────────┼────────────────────────┐
         │                        │                        │
┌────────▼────────┐      ┌────────▼────────┐      ┌────────▼────────┐
│   x86_64 AVX2   │      │   ARM64 NEON    │      │    FASM x64     │
│   256-bit FMA   │      │   128-bit FMA   │      │ Bare-Metal ASM  │
│ (32 floats/iter)│      │ (16 floats/iter)│      │  (Windows x64)  │
└─────────────────┘      └─────────────────┘      └─────────────────┘

1. 256-bit AVX2 + FMA (src/nanovector_avx2.c):
   • 4-way unrolled kernel processing 32 single-precision floats per loop iteration across 4 YMM accumulators.
   • Fused multiply-accumulate (_mm256_fmadd_ps) eliminates intermediate register spills.
   • Tail handling handles arbitrary vector dimensions with zero padding penalties.
2. ARM NEON (src/nanovector_neon.c):
   • 128-bit vectorization for Apple Silicon (M1/M2/M3/M4) and AWS Graviton processors.
   • 4-way unrolling processing 16 floats per iteration using vfmaq_f32 and vaddvq_f32.
3. Pure FASM Assembly (src/asm/nanovector_x64.asm):
   • Hand-crafted Windows x64 assembly routines adhering strictly to Microsoft x64 ABI calling conventions (volatile register allocation ymm0..ymm5, shadow store handling).
   • Assembles cleanly into a 629-byte object file using Flat Assembler (FASM).
4. In-Place Top-$K$ Heap:
   • Min-heap / Max-heap maintains the best $K$ matches in $O(N \log K)$.
   • Branch-predicted pruning: candidate items with scores worse than the current $K$-th element are discarded in a single CPU clock cycle.
5. .nvec Binary Specification:
   • 64-byte aligned header with magic bytes NVEC\x01.
   • Contiguous $N \times D \times 4$ raw float block (zero-copy memory-mappable).
   • Compact length-prefixed ID and JSON metadata string tables.

🐍 Python API Reference

nanovector.Index(dim: int, metric: str = "cosine", normalize: bool = False)

Initializes an embedded vector index.

• dim (int): Vector dimensionality (e.g. 384, 768, 1536).
• metric (str): Distance metric:
  • "cosine": Cosine similarity ($\frac{u \cdot v}{|u| |v|}$), higher is closer. Range $[-1.0, 1.0]$.
  • "dot" or "ip": Inner Product ($u \cdot v$), higher is closer.
  • "l2" or "euclidean": Squared Euclidean distance ($\sum (u_i - v_i)^2$), lower is closer.
• normalize (bool): If True, vectors are automatically L2-normalized upon insertion and search.

Methods

Properties

• index.dim (int): Dimensionality of indexed vectors.
• index.count (int) or len(index): Total number of indexed vectors.
• index.metric (str): Active distance metric.
• nanovector.version() (str): Library version string (e.g. "0.1.0").
• nanovector.simd_backend() (str): Active hardware acceleration backend ("AVX2+FMA (x86_64)", "ARM NEON", etc.).

🌐 High-Performance Systems Ecosystem

nanovector is developed by @eminsk as part of an open-source performance ecosystem:

• ⚡ NanoGEMM — Bare-metal AVX2+FMA SIMD matrix multiplication engine in ~100KB for sub-microsecond CPU neural network inference (pip install nanogemm).
• 📈 yfinance-ta-patterns — Institutional-grade technical pattern scanner with AI Confluence Scoring and LLM prompt generation (pip install yfinance-ta-patterns).
• 🎥 screenvideo — Desktop screen recorder with WASAPI audio and standalone pure x64 FASM edition.
• 📊 xlsx_vievers — Desktop spreadsheet processor with SSE2 SIMD hardware math engine.
• 🔍 StackOverflowAPI — Bilingual desktop client with native FASM x64 search client.

📄 License

MIT License. See LICENSE for details.