Routing Memory (RM)

Lightweight long-term memory for LLM agents via vector-quantized routing.

RM replaces brute-force dense retrieval with a VQ codebook that compresses N items into K centroid buckets. Queries probe only the top-n centroids, rerank by dot product, and return results — achieving 768x per-item compression and 99%+ recall at a fraction of the latency and memory cost.

pip install routing-memory

Quick Start

from rm import RoutingMemory

memory = RoutingMemory()

# Store memories
memory.add("User prefers dark mode for all applications")
memory.add("Meeting with Alice scheduled for March 15 at 2pm")
memory.add("Project deadline is end of Q1 2026")

# Search
results = memory.search("what are the user's UI preferences?", top_k=3)
for r in results:
    print(f"  [{r['score']:.3f}] {r['text']}")

Features

Architecture

Query ──> Encode ──> Top-n Centroids ──> Collect Candidates ──> Dot-Product Rerank ──> Filter ──> Results
                          |                                           |
                     VQ Codebook                               Score Threshold
                     (K centroids)                               (tau >= 0.3)

Compression: Each item needs only a 2-byte centroid assignment vs 384x4 = 1536 bytes for dense fp32. That's 768x compression.

Recall: With n=4 probes on 5K items: R@5 = 0.9916 (99.2% of dense baseline).

API Reference

RoutingMemory

RoutingMemory(
    db_path="rm_memory.db",      # SQLite path (None for in-memory)
    embedding_model="all-MiniLM-L6-v2",  # any sentence-transformers model
    n_probes=3,                  # centroids to probe per query
    score_threshold=0.3,         # minimum retrieval score
    seed=42,                     # random seed
)

Methods:

Low-level Components

from rm import Codebook, L1Retriever, DriftMonitor

# Direct codebook access
cb = Codebook(dim=384, seed=42)
cb.fit(embeddings, item_ids)
centroid_id, qerr = cb.encode(query_embedding)
conf = cb.conf(query_embedding)
margin = cb.margin(query_embedding)

# Retriever
retriever = L1Retriever(cb, n_probes=4, top_k=10, score_threshold=0.3)
result = retriever.query(query_embedding)  # returns L1Result

# Drift monitor
monitor = DriftMonitor()
alarm = monitor.record(qerr, margin)  # returns DriftAlarm or None

Experiment Suite

RM ships with 13 reproducible experiments (7 hypothesis tests + 6 application benchmarks).

# Run all experiments
python -m rm.experiments.run_all

# Run specific experiments
python -m rm.experiments.run_all --select H1 H2 A4

Results Summary

All experiments use real embeddings (all-MiniLM-L6-v2, d=384) and seed=42.

Project Structure

rm/
  rm/                     # Core package
    __init__.py
    codebook.py           # VQ codebook (MiniBatchKMeans, adaptive K)
    retrieval.py          # Multi-probe retrieval with dot-product rerank
    filtering.py          # Score-based result filtering
    drift.py              # Distribution drift detection
    memory.py             # RoutingMemory high-level API
    embeddings/           # Pluggable embedding backends
      base.py             # Abstract interface
      local.py            # sentence-transformers wrapper
    storage/              # Pluggable storage backends
      base.py             # Abstract interface
      sqlite.py           # SQLite persistence
  experiments/            # 13 reproducible experiments
    run_all.py            # Experiment runner (--select support)
    shared/               # Data generation, plotting utilities
    h1_codebook/ .. h7_storage/   # Hypothesis tests
    a1_msmarco/ .. a6_imbalance/  # Application benchmarks
  tests/                  # pytest test suite
  pyproject.toml          # Package configuration
  LICENSE                 # MIT

Development

git clone https://github.com/AhmetYSertel/routing-memory.git
cd routing-memory
pip install -e ".[dev]"
pytest tests/ -v

Citation

If you use RM in your research, please cite the HGA paper:

@article{sertel2026hga,
  title={Hybrid Governance Architecture: Structured Memory and Adaptive Routing for LLM Agents},
  author={Sertel, Ahmet Yigit},
  year={2026}
}

License

MIT