synaptic-memory 0.17.2

Knowledge graph + MCP tool server for LLM agents — 3rd-gen retrieval, graph-aware multi-turn exploration, structured data tools, DB→ontology one-liner.

Synaptic Memory

Zero API calls at index time. Zero infra. Zero lock-in. A knowledge graph + MCP tool server for LLM agents, with hybrid retrieval, CDC-based live database sync, and Korean FTS built in.

한국어 README

---

5-minute start

pip install "synaptic-memory[sqlite,korean,vector]"
python examples/quickstart.py

That command ingests examples/data/products.csv into a SQLite-backed graph and runs three searches — all without calling any LLM at indexing time. Full source: examples/quickstart.py.

---

Two calls to build a graph

import asyncio
from synaptic import SynapticGraph

async def main():
    # Any data → knowledge graph (CSV, JSONL, directory)
    graph = await SynapticGraph.from_data("./my_data/")

    # Or directly from a database — SQLite / PostgreSQL / MySQL / Oracle / MSSQL
    graph = await SynapticGraph.from_database(
        "postgresql://user:pass@host:5432/dbname"
    )

    # Live database? Use CDC mode and only re-read what changed.
    graph = await SynapticGraph.from_database(
        "postgresql://user:pass@host:5432/dbname",
        db="knowledge.db",
        mode="cdc",       # deterministic node IDs + sync state recorded
    )
    result = await graph.sync_from_database(
        "postgresql://user:pass@host:5432/dbname"
    )
    print(result.added, result.updated, result.deleted)

    # Or bring your own chunker (LangChain, Unstructured, custom OCR, ...)
    chunks = my_parser.split("manual.pdf")
    graph = await SynapticGraph.from_chunks(chunks)

    # Search
    result = await graph.search("my question", engine="evidence")

asyncio.run(main())

That's it. Auto-detects file format or DB schema, generates an ontology profile, ingests, indexes, builds FK edges.

Live database sync (CDC) — mode="cdc" enables incremental updates: tables with an updated_at-style column are read with a watermark filter, others fall back to per-row content hashing. Deletes are detected via a TEMP TABLE LEFT JOIN; FK rewires re-link the corresponding RELATED edges. Search results are identical to a full reload (locked in by a regression test). Supports SQLite, PostgreSQL, MySQL/MariaDB.

Office files (PDF/DOCX/PPTX/XLSX/HWP) are supported through the optional xgen-doc2chunk package. Install with pip install synaptic-memory[docs] or use from_chunks() with your own parser.

---

What it does

Your data (CSV, JSONL, PDF/DOCX/PPTX/XLSX/HWP, SQL database)
  ↓  auto-detect format / auto-discover DB schema + FKs
  ↓  DocumentIngester (text) / TableIngester / DbIngester
  ↓
Knowledge Graph
  ├─ Documents: Category → Document → Chunk
  └─ Structured: table rows as ENTITY nodes + RELATED edges (FKs)
  ↓
36 MCP tools → LLM agent explores via graph-aware multi-turn tool use

Two jobs, nothing else:

1. Build the graph well — cheap extraction, no LLM at index time
2. Give the LLM good tools — the agent decides what to search

---

Install

# Recommended — covers every example in this README
pip install "synaptic-memory[sqlite,korean,vector,mcp]"

# Or everything, including Postgres / Kuzu / Qdrant / MinIO
pip install "synaptic-memory[all]"

Pick-your-own extras

pip install synaptic-memory                # Core (zero deps, in-memory only)
pip install synaptic-memory[sqlite]        # + SQLite FTS5 backend
pip install synaptic-memory[korean]        # + Kiwi morphological analyzer
pip install synaptic-memory[vector]        # + usearch HNSW index
pip install synaptic-memory[mcp]           # + MCP server for Claude
pip install synaptic-memory[embedding]     # + aiohttp for embedding APIs
pip install synaptic-memory[reranker]      # + flashrank cross-encoder
pip install synaptic-memory[postgresql]    # + asyncpg + pgvector
pip install synaptic-memory[docs]          # + xgen-doc2chunk (PDF/DOCX/PPTX/XLSX/HWP)

---

Quick Start

Option A: Two lines (easiest)

import asyncio
from synaptic import SynapticGraph

async def main():
    # CSV file
    graph = await SynapticGraph.from_data("products.csv")

    # JSONL documents
    graph = await SynapticGraph.from_data("documents.jsonl")

    # Entire directory (scans all CSV/JSONL)
    graph = await SynapticGraph.from_data("./my_corpus/")

    # With embedding (optional, improves semantic search)
    graph = await SynapticGraph.from_data(
        "./my_corpus/",
        embed_url="http://localhost:11434/v1",
    )

    # Search
    result = await graph.search("my question", engine="evidence")
    for activated in result.nodes[:5]:
        print(activated.node.title, activated.activation)

asyncio.run(main())

Option B: MCP server (Claude Desktop / Code)

synaptic-mcp --db my_graph.db
synaptic-mcp --db my_graph.db --embed-url http://localhost:11434/v1

Claude can now call 36 tools to explore your graph — search, ingest new files into the graph mid-conversation, and sync from a live database without dropping to a CLI.

A ready-to-paste claude_desktop_config.json snippet is in examples/mcp_claude_desktop.json.

Option BX: LangChain retriever (drop-in)

pip install "synaptic-memory[sqlite,korean,vector,langchain]"

import asyncio
from synaptic import SynapticGraph
from synaptic.integrations.langchain import SynapticRetriever

async def main():
    graph = await SynapticGraph.from_data("./docs/")
    retriever = SynapticRetriever(graph=graph, k=5, engine="evidence")

    docs = await retriever.ainvoke("my question")
    for doc in docs:
        print(doc.page_content[:80], "   ", doc.metadata["score"])

asyncio.run(main())

Runnable example: examples/langchain_retriever.py. Each hit becomes a LangChain Document with the node id, title, score, and any structured properties in metadata — works unmodified in RetrievalQA chains, agents, and RAG graphs.

Option C: Full control

import asyncio
from synaptic.backends.sqlite_graph import SqliteGraphBackend
from synaptic.extensions.domain_profile import DomainProfile
from synaptic.extensions.document_ingester import DocumentIngester, JsonlDocumentSource

async def main():
    profile = DomainProfile.load("my_profile.toml")
    backend = SqliteGraphBackend("graph.db")
    await backend.connect()

    source = JsonlDocumentSource("docs.jsonl", "chunks.jsonl")
    ingester = DocumentIngester(profile=profile, backend=backend)
    await ingester.ingest(source)

asyncio.run(main())

---

Indexing cost comparison

Approach	LLM at indexing	Trade-off
GraphRAG-style (MS GraphRAG, Cognee, Graphiti)	LLM extracts entities + relations + community summaries	Highest recall on narrative corpora, but every new document costs LLM tokens
LightRAG-style	LLM deferred to query time	Less index cost, but each query pays
Synaptic	None. Structural + statistical signals only (FK, NEXT_CHUNK, phrase DF hubs, MENTIONS)	Cheapest, deterministic, but won't synthesize new relations on its own

No LLM at indexing. The graph is a search index, not a knowledge base. If you need LLM-synthesized summaries on top of the graph, layer them with your own agent — Synaptic gives you the primitives and leaves the synthesis choice to you.

v0.16.0+: graph.search() defaults to the hybrid EvidenceSearch pipeline (BM25 + HNSW + PPR + cross-encoder + MMR). engine="legacy" still works but raises DeprecationWarning; removal is pushed to v0.18.0 to bundle with HippoRAG2-style architecture work.

---

Agent Tools (36 total)

Text search tools

Tool	Purpose
deep_search	Recommended. Search → expand → read documents in ONE call
compare_search	Auto-decompose multi-topic queries, search in parallel
knowledge_search	Core semantic search (routes through EvidenceSearch in v0.14.2+)
agent_search	FTS + vector hybrid search with intent routing
expand	1-hop graph neighbours
get_document	Full document with query-relevant chunks
search_exact	Literal substring match for IDs/codes
follow	Walk a specific edge type

Structured data tools

Tool	Purpose
filter_nodes	Property filter (>=, <=, contains) — returns {total, showing} for accurate counting
aggregate_nodes	GROUP BY + COUNT/SUM/AVG/MAX/MIN with optional WHERE pre-filter
join_related	FK-based related record lookup — walks RELATED edges (O(degree))

Ingest / CDC tools (v0.14.0+)

Mid-conversation ingestion so Claude can teach itself new material without leaving the chat.

Tool	Purpose
knowledge_add_document	Ingest a long-text document with automatic sentence-boundary chunking
knowledge_add_table	Ingest structured rows → ENTITY nodes + FK edges
knowledge_add_chunks	BYO-chunker path for pre-split content
knowledge_ingest_path	Ingest a CSV / JSONL / text file from the local filesystem
knowledge_remove	Delete a single node with edge cascade
knowledge_sync_from_database	Incremental sync from a live database (CDC)
knowledge_backfill	Repair graphs missing embeddings or phrase hubs (v0.14.4+)

Navigation tools

Tool	Purpose
list_categories	Category list with document counts
count	Structural count by kind/category/year
session_info	Multi-turn session state

All tools return { data, hints, session }. The SearchSession tracks seen nodes across turns so the agent never re-reads the same chunk.

---

Retrieval Pipeline

Query
  ↓  Kiwi morphological analysis (Korean) or regex (other)
  ↓  BM25 FTS + title 3x boost + substring fallback
  ↓  Vector search (usearch HNSW, optional)
  ↓  Vector PRF (pseudo relevance feedback, 2-pass)
  ↓  PPR graph discovery (personalized pagerank)
  ↓  GraphExpander (1-hop: category siblings, chunk-next, entity mentions)
  ↓  HybridReranker (lexical + semantic + graph + structural + authority + temporal)
  ↓  MaxP document aggregation (coverage bonus)
  ↓  Cross-encoder reranker (bge-reranker-v2-m3 via TEI, optional)
  ↓  EvidenceAggregator (MMR diversity + per-doc cap + category coverage)
Result

---

Benchmarks

Reproducible FTS-only baseline (< 2 seconds on a laptop)

pip install "synaptic-memory[korean]"
python examples/benchmark_allganize.py

Output (deterministic, v0.16.0):

Dataset                  Corpus  Queries      MRR     R@10        Hit     Time
--------------------------------------------------------------------------------
Allganize RAG-ko            200      200    0.947    1.000   200/200     9.3s
Allganize RAG-Eval          300      300    0.911    0.950   285/300     5.9s

This is the embedder-free baseline (EvidenceSearch pipeline: BM25 + PPR + MMR, no vector index, no cross-encoder). Full source: examples/benchmark_allganize.py. Data source: allganize/RAG-Evaluation-Dataset-KO.

v0.16.0 — engine default flipped to "evidence". Combined with the v0.15.1 query-mode Kiwi improvement, FTS-only Korean retrieval moved from Allganize RAG-ko MRR 0.621 (v0.15.0) → 0.947 (v0.16.0) without any embedder or reranker. English (HotPotQA-24) 0.727 → 0.875. Full ablation: examples/ablation/run_ablation.py. Reproducibility under streaming ingest sharpens in lockstep: top-1 rank invariance rose from 54.5 % → 100 %, bit-wise top-10 identical from 51.5 % → 96 %, with MRR drift exactly zero.

Head-to-head vs Mem0 / Cognee / HippoRAG2

A runnable harness (BEIR-style corpus, same MRR / R@10 scoring code for every system):

# Synaptic only (no API keys, ~2 s)
python examples/benchmark_vs_competitors/run_comparison.py --only synaptic

# Adapters for Mem0, Cognee, HippoRAG2 ship in-tree — add them to
# --only when the respective packages are installed and API keys are set
python examples/benchmark_vs_competitors/run_comparison.py --only synaptic,mem0 --subset 10

See examples/benchmark_vs_competitors/README.md for fairness caveats. Competitor self-reported numbers (Mem0 LoCoMo 91.6, HippoRAG2 MuSiQue F1 51.9, the Zep 84→58 correction incident, etc.) are catalogued with sources in docs/comparison/published_numbers.md.

Embedder-free full-dataset summary (v0.16.0)

Run via python examples/ablation/run_ablation.py:

Dataset	Lang	Queries	MRR	Hit @ 10
Allganize RAG-ko	ko	200	0.947	200/200
Allganize RAG-Eval	ko	300	0.911	285/300
AutoRAG KO	ko	114	0.906	114/114
PublicHealthQA KO	ko	77	0.546	64/77
HotPotQA-24 EN	en	24	0.875	24/24

English multi-hop standard benchmarks (v0.16.0, subset)

Run via:

pip install "synaptic-memory[eval]"   # adds `datasets` for HuggingFace download
python examples/ablation/download_benchmarks.py
python examples/ablation/run_tier1_benchmarks.py --subset 500

Adds HotPotQA-dev (66 k corpus), MuSiQue-Ans-dev (21 k), and 2WikiMultihopQA-dev (57 k) — the three retrieval corpora the HippoRAG / GraphRAG line of research uses for head-to-head. Numbers go in docs/comparison/synaptic_results.md.

Full pipeline (BGE-M3 + cross-encoder) — v0.17.1

Measured 2026-04-19 on an H100, BAAI/bge-m3 + BAAI/bge-reranker-v2-m3 loaded in-process via transformers (no external TEI / Ollama endpoint needed):

python eval/run_all.py --quick --local-bge

14-bench single-shot (5 public + 9 private)

Dataset	Lang	Queries	FTS-only	Full pipeline	Δ
HotPotQA-24	en	24	0.875	0.979	+0.104
Allganize RAG-ko	ko	200	0.947	0.983	+0.036
Allganize RAG-Eval	ko	300	0.911	0.955	+0.044
PublicHealthQA	ko	77	0.547	0.748	+0.201
AutoRAG KO	ko	114	0.906	0.806	−0.100 ⚠
KRRA Easy	ko	20	0.967	0.975	+0.008
KRRA Hard	ko	40	0.583	0.589	+0.006
KRRA Conv	ko	30	0.146	0.166	+0.020
assort Easy	ko	15	0.760	0.856	+0.096
assort Hard	ko	40	0.000	0.000	0
assort Conv	ko	30	0.425	0.472	+0.047
X2BEE Easy	en	20	1.000	1.000	0
X2BEE Hard	en/ko	20	0.379	0.368	−0.011
X2BEE Conv	en/ko	30	0.167	0.164	−0.003
Mean			0.615	0.647	+0.032 (+5.2pp)

v0.17.1 is the first release where mean Full-pipeline MRR exceeds mean FTS-only MRR across all 14 benches (v0.17.0 was net −1.1 %). 12/14 benches improve or hold; the 3 mild residual regressions (X2BEE Hard / Conv, AutoRAG) are at or below single-query noise except for AutoRAG where the structural reranker mismatch persists.

When the cross-encoder helps — and when it hurts. The reranker shines on paraphrase-heavy corpora (PublicHealthQA +20 pp, HotPotQA +10 pp). On retrieval-style corpora where FTS ranking is already near-optimal (AutoRAG, X2BEE Hard) it injects noise that displaces the gold rank. v0.17.1's adaptive blend ( std/3 discriminator) and structured-row reranker skip recover most of that — AutoRAG went −0.264 → −0.100 — but pure reranker=None is the only way to hit the FTS-only ceiling on those corpora. Diagnostic: examples/ablation/diagnose_autorag.py.

Multi-turn agent (Qwen3.5-27B via vLLM, 5 turns, LLM-judge)

Dataset	Single-shot	Agent solved	vs v0.13 (gpt-4o-mini)
KRRA Hard	0.589	30/39 (77 %)	11/15 (73 %) +4 pp
assort Hard	0.000	30/33 (91 %)	13/15 (87 %) +4 pp
X2BEE Hard	0.368	19/19 (100 %)	17/19 (89 %) +11 pp
KRRA Conv	0.166	14/30 (47 %)	21/30 (70 %) −23 pp
assort Conv	0.472	22/24 (92 %)	20/24 (83 %) +9 pp
X2BEE Conv	0.164	25/27 (93 %)	22/27 (81 %) +12 pp
Mean		140/172 = 81 %

This is Synaptic's real number. Single-shot retrieval is the floor; the multi-turn agent (deep_search + compare_search + graph-context injection) brings the same questions from 0–47 % to 47–100 %. assort Hard 0/40 → 91 % under agent shows what the graph + structured tools can do that no single-shot pipeline can. 5 of 6 benches beat the v0.13 GPT-4o-mini baseline (Qwen3.5-27B upgrade). KRRA Conv regression (Qwen Korean conversational gap) is the one open issue — v0.18 track.

Known structural gap — MuSiQue (English multi-hop)

MuSiQue-Ans-dev 500q full pipeline R@5 0.453 vs HippoRAG2's published 0.747 (−0.294). Three rounds of targeted fixes (LLM query decomposition, inline phrase hub, DF-filtered entity linker) all regressed the score — the gap is structural. Closing it requires OpenIE triple extraction + query→triple dense linking, which is a v0.18.0+ research track rather than a default pipeline change. Synaptic's strength is Korean / structured-data RAG; English Wikipedia multi-hop is honestly documented as a trade-off. See docs/PLAN-v0.18-architecture.md.

Multi-turn agent (GPT-4o-mini, 5 turns max)

Dataset	Result
KRRA Hard agent	10-13/15 (67-87%)
X2BEE Hard agent	17/19 (89%)
assort Hard agent	12/15 (80%)

Structured data queries (filter / aggregate / FK join / count) work end-to-end through graph-aware tools.

---

Architecture

SynapticGraph.from_data("./data/")          ← Easy API
  ↓
Auto-detect → DomainProfile → Ingest → Index
  ↓
StorageBackend (Protocol)
  ├── MemoryBackend        (testing)
  ├── SqliteGraphBackend   (recommended, FTS5 + HNSW)
  ├── KuzuBackend          (embedded Cypher)
  ├── PostgreSQLBackend    (pgvector)
  └── CompositeBackend     (mix backends)
  ↓
Retrieval pipeline (BM25 + vector + PRF + PPR + reranker + MMR)
  ↓
Agent tools (36) → MCP server → LLM agent

---

Backends

Backend	Vector Search	Scale	Use Case
MemoryBackend	cosine	~10K	Testing
SqliteGraphBackend	usearch HNSW	~100K	Default
KuzuBackend	HNSW	~10M	Graph-heavy
PostgreSQLBackend	pgvector	~1M	Production
CompositeBackend	Qdrant	Unlimited	Scale-out

---

Optional Extras

Extra	What it adds
korean	Kiwi morphological analyzer for Korean FTS
vector	usearch HNSW index (100x faster vector search)
embedding	aiohttp for embedding API calls
mcp	MCP server for Claude Desktop/Code
sqlite	aiosqlite backend
docs	xgen-doc2chunk for PDF/DOCX/PPTX/XLSX/HWP loading

---

Documentation

Doc	What it is
docs/GUIDE.md	Friendly intro — what/why/how, zero jargon (Korean)
docs/TUTORIAL.en.md	30-minute hands-on walkthrough (English)
docs/TUTORIAL.md	30-minute hands-on walkthrough (Korean)
docs/CONCEPTS.md	3rd-gen GraphRAG + pipeline internals
docs/ARCHITECTURE.md	Original neural-inspired design
docs/COMPARISON.md	vs GraphRAG / LightRAG / LazyGraphRAG
docs/comparison/synaptic_results.md	Reproducible Synaptic numbers with provenance
docs/comparison/published_numbers.md	Competitor self-reported numbers (with sources)
docs/paper/draft.md	arXiv preprint draft — Streaming Retrieval with Top-K Invariance
docs/paper/theorem.md	Formal theorem + proof sketch
docs/ROADMAP.md	Future plans

Dev

uv sync --extra dev --extra sqlite --extra mcp
uv run pytest tests/ -q                   # 809+ tests
uv run ruff check --fix

License

Apache-2.0 — see LICENSE. Permits commercial use, modification, and redistribution as long as the copyright/attribution notice is preserved.