mda-memory 0.1.2

Online associative memory for LLMs

MDA — Modular Dynamic Architecture

Online associative memory for LLMs. Learns during inference. No backpropagation. No GPU required.

---

What is MDA?

Large language models can reason but cannot remember. RAG partially addresses this — but it cannot update during a conversation, and it cannot learn from the conversation itself.

MDA fills precisely these gaps.

It encodes knowledge as 256-dimensional Holographic Distributed Representations (HDRs), connects concepts through a sparse synapse graph, and retrieves context by activating entity networks — not by text-chunk similarity search. New knowledge is integrated immediately, without rebuilding any index.

MDA is not a RAG replacement. It is the persistent learning layer that RAG and LLMs are missing.

---

Key Properties

• Token-free — no tokenizer, no vocabulary
• Attention-free — no transformer encoder required
• Online learning — learns during inference via the Oja rule
• No catastrophic forgetting — entities are independent; new knowledge never overwrites old
• CPU-only — runs entirely on numpy, no GPU required
• Model-agnostic — works with Ollama, OpenAI, Anthropic, or any LLM

---

Benchmark Results

Evaluated against a strong RAG baseline (bge-large-en-v1.5 + ChromaDB, top-6 retrieval) across 80 questions spanning 8 cognitive categories:

Category	RAG	MDA	Δ
ATOMIC_RECALL	100%	85%	−15%
MULTI_HOP	90%	90%	0%
CROSS_DOCUMENT	80%	70%	−10%
REASONING	70%	90%	+20%
INCREMENTAL_LEARNING	0%	60%	+60%
NOISE_RESISTANCE	100%	100%	0%
MEMORY_COMPRESSION	90%	70%	−20%
BOUNDARY	100%	100%	0%
OVERALL	78.8%	83.1%	+4.3%

MDA uses 3.1× less context per query than RAG while achieving higher overall accuracy.

Long-context retention (200 turns): RAG 0% — MDA 92%.

---

Quick Start

pip install mda-memory

Or clone for development:

git clone https://github.com/Rangle2/mda
cd mda
pip install -e .

Basic Usage

from mda import MDA

memory = MDA()

# Teach facts
memory.learn("The capital of Veloria is Aranthos.")
memory.learn("Aranthos was founded by Queen Seraphel in 412 AE.")

# Retrieve context — returns memory string ready for LLM injection
context = memory.context_for("Who founded the capital?")
print(context)
# → [MEMORY] Aranthos was founded by Queen Seraphel in 412 AE.

With Ollama

# Start Ollama with any model
ollama pull qwen3:4b

# Run MDA CLI
mda --model qwen3:4b

With Anthropic (Claude)

# Set your API key
export ANTHROPIC_API_KEY=your_key_here

# Run MDA CLI with Claude
mda --model claude-haiku-4-5-20251001 --provider anthropic

Running Benchmarks

# Main benchmark — Ollama
python benchmark/benchmark.py --model qwen3:4b

# Main benchmark — Anthropic
python benchmark/benchmark.py --model claude-haiku-4-5-20251001 --provider anthropic

# Long-context retention benchmark — Ollama
python benchmark/long_context_benchmark.py --model qwen3:4b

# Long-context retention benchmark — Anthropic
python benchmark/long_context_benchmark.py --model claude-haiku-4-5-20251001 --provider anthropic

---

Project Structure

mda/
├── mda/
│   ├── __init__.py        # Public API — exposes MDA class
│   ├── mda.py             # MDA class implementation
│   ├── core/
│   │   ├── bind.py        # HDR ops: bind, unbind, normalize, cosine
│   │   ├── encoder.py     # HolisticEncoder: text → 256-dim vector
│   │   ├── entity.py      # Entity: v, r, h, W, neurons, synapses, senses
│   │   ├── neuron.py      # Neuron (Oja rule), Synapse (Hebbian)
│   │   └── registry.py    # EntityRegistry: surface → Entity lookup
│   ├── inference/
│   │   ├── associative.py # AssociativeChain: multi-hop traversal
│   │   ├── broca.py       # BrocaModule: W-hybrid context scoring
│   │   ├── reasoning.py   # ReasoningEngine: cross-entity inference
│   │   └── memory.py      # ConversationMemory: multi-turn context
│   ├── training/
│   │   └── checkpoint.py  # save/load: float32, atomic write
│   └── integrations/
│       ├── engine.py      # MDAEngine: LLM bridge (Ollama / Anthropic)
│       └── cli.py         # Interactive CLI (Rich + prompt_toolkit)
├── benchmark/
│   ├── benchmark.py       # 80-question main benchmark
│   └── long_context_benchmark.py  # 200-turn retention benchmark
├── tests/                 # pytest test suite
└── pyproject.toml

---

How It Works

Entity & W Matrix

Every concept is an Entity with a 256-dim identity vector v and a lazy-initialized Hebbian weight matrix W. W is None until the entity is first activated — memory overhead is proportional to usage.

Online Learning (Oja Rule)

ΔW = η(yxᵀ − y²W)

where x is the query vector and y = Wx. No backpropagation. No gradient descent. Runs in O(d²) per entity per turn.

AssociativeChain

Query → origin entity → breadth-first synapse traversal (depth 3) → context assembly. Dynamic inhibition prevents context flooding.

BrocaModule

Facts are scored with a W-hybrid formula:

score = 0.35·s_query + 0.45·s_W + 0.20·s_sense

Facts below 0.20 are suppressed.

---

Running Tests

pytest tests/

---

Roadmap

• GPU acceleration — PyTorch tensor ops, parallel entity traversal
• Low-rank W — W ≈ A×B factorization for higher-dimensional HDRs
• mda.cloud API — persistent memory as a service
• MDA + RAG hybrid — offline corpus retrieval + online conversational learning
• Real-world corpus evaluation — beyond fictional benchmark domains
• Independent benchmark — community-constructed evaluation set

---

License

SSPL 1.0 — free for research and personal use. Commercial use requires a separate agreement.

For commercial licensing: [e-mail]