yadgar 5.192.0

Changelog · Benchmark · Architecture · Roadmap · JS/TS SDK · Agents guide · CLI cheat-sheet

AI coding agents: the operational guide lives in AGENTS.md — setup commands, dev environment, test runner, code style, PR rules, security gates. This README is the human overview.

Why this repo lives on GitHub (moved from Codeberg, 2026): Codeberg's updated Terms of Use — § 2 (1) 7, added in "Proposal Assembly 2026: prohibit LLM-extrusions" (2026-06-29) — now ask that people not host projects that "mostly consist of code written by 'generative AI'-tools." Yadgar is built with heavy AI-agent assistance and is candid about it, so it no longer fits Codeberg's hosting policy and has moved here. No hard feelings — Codeberg is an excellent home for human-authored free software; this is a policy-fit decision, nothing more.

A persistent memory engine for MCP clients. Tell it what matters and it survives across sessions — decaying what you stop touching, promoting what repeats, scoping recall to the project you're in, and pairing every memory with a curated wiki that searches through the same ranking pipeline.

Supported clients: one shared streamable-HTTP daemon (http://127.0.0.1:8765/mcp) serves the memory and wiki MCP surface to all 9 supported clients: claude-code, codex, gemini, cursor, cline, windsurf, kiro, amp, opencode. Claude Code additionally gets the full harness integration (hooks, task-list mirror, CLAUDE.md sync). All other clients receive MCP registration and a rules file (AGENTS.md-equivalent).

Multi-client setup: use yadgar install to wire any client. Register a single client by name, or probe and register all detected clients at once:

yadgar install --client <name>           # e.g. --client opencode
yadgar install --auto-detect             # detect + register all installed clients
yadgar install --client <name> --print   # dry-run: emit JSON to stdout, no file writes

--print is the nix/home-manager contract (task #67): it outputs the full config without touching the filesystem, suitable for declarative activation. Per-platform detail and flag reference: docs/reference/install.md.

Yadgar (یادگار) is Persian for "memento, keepsake."

What it is

Claude Code forgets everything when a session ends, when you /clear, or when context compacts. Yadgar is the layer that remembers. It runs as an MCP server alongside Claude Code and gives it two durable stores:

1. Memory — episodic + semantic facts, each carrying a heat value that decays over time. Access reheats; disuse decays; recurring episodes get promoted to semantic memory by a nightly consolidation cycle modelled on how brains sleep.
2. A curated wiki — long-lived, versioned, project-scoped pages: conventions, module purpose, past decisions, where subsystems live. Written deliberately, not captured incidentally.

A single recall() query searches both stores at once, fuses and re-ranks the results, weights them by heat, gates out decayed entries, and scopes them to your project. Critical context can be anchored so it never decays; whole working states can be checkpointed and restored across a /compact.

Why

• Survives sessions. Heat decay drops what you stopped using; surprise gating drops duplicates on arrival; anchors pin what must never be lost.
• Project-scoped. Every memory and wiki page is owned by a project_id (owner/repo), and recall resolves against it: your project's rows first, then the global-reach library. Branch scoping was removed in ADR-0215 — nothing filters or boosts on the git branch any more, and the branch column is gone (migrations 029 + 032).
• One ranking pipeline for memory + wiki. Curated knowledge and episodic memory come back in a single ranked result, not two separate searches.
• Consolidates while you sleep. A nightly brain cycle decays heat, promotes episodes to semantics, discovers causal links, forms associative links, and merges duplicates — without dropping the MCP connection.
• Self-documenting. Architecture Decision Records, a reusable agent-prompt library, a harness task-list mirror, and an interactive 3D knowledge-graph all live in the same store.

Key features

Memory

• Heat-decay lifecycle — every memory carries heat ∈ [0,1] decaying exponentially; access reheats, disuse decays, unused items eventually archive and purge.
• Surprise-gated writes — a similarity gate rejects near-duplicate writes on arrival (rejections land in the DLQ, inspectable via dlq_inspect).
• Write-time contradiction detection — a lightweight negation + action-divergence heuristic flags contradicting memories and decays their confidence, without blocking the write.
• Anchors — protected memories that never decay; audit_anchors() surfaces redundant, oversize, expired, or near-duplicate (cosine ≥ 0.95) anchors across projects.
• Checkpoint / restore — checkpoint(directory) snapshots your working state before compaction; restore(directory) reconstructs it afterward via a hippocampal-replay blend of checkpoint + anchors + hot memories + predicted context. Use restore, not recall, to resume after /clear or /compact.
• Memory blocks — Letta-style named, scoped, char-limited core-memory containers (block_*) for always-in-context working state.

Curated wiki

• Versioned pages — full history (wiki_history, wiki_diff, wiki_read_version, wiki_restore); wiki_add commits directly (no draft step).
• Directory-scoped resolution — wiki_read(slug) resolves caller directory → global → not found. (Branch scoping was removed by ADR-0215.)
• Surgical editing — anchor-text and positional edit tools (wiki_replace_text, wiki_insert_before/after, wiki_append_section, wiki_replace_markdown_block, positional wiki_*_at) so pages mutate in place instead of full rewrites.
• Auto-linking — wiki_autolink inserts [[slug]] cross-references by matching page titles; validated so it never manufactures broken refs.
• Sync helpers — wiki_refresh_stale, wiki_coverage, wiki_lint.
• Code graph (on by default, opt-out; successor to the retired repo-wiki generator) — yadgar code-graph shells out host-side to the codebase-memory-mcp static binary (158-language tree-sitter, offline) to index the latest origin/<default-branch> and render a bounded architecture digest into an always-injected memory block (recall-free). code_graph.enabled defaults to true in the DB-backed runtime-config store (no row needed) and every installer surface installs the host binary automatically — unattended, no prompt, so a scripted/QA install needs no flags. yadgar setup, yadgar-setup, and make setup all route through yadgar code-graph install. Each has one opt-out — yadgar setup --no-code-graph, yadgar-setup --no-code-graph, make setup YADGAR_CODE_GRAPH=0 — which skips the binary and persists code_graph.enabled=false, so the flag and the binary can never disagree (a failed download, e.g. offline, does the same rather than aborting the install). Opt a single repo out with config_set("code_graph.enabled", false, scope="project", directory=<repo>), or disable globally with config_set("code_graph.enabled", false, scope="global") — a per-dir override always wins over global. See ADR-0162/0163.
• Bookmarks — pin wiki pages in the viz UI (bookmark_*); drag-to-reorder, dense-integer positions.

Unified recall

• One recall() query retrieves memory + wiki together, fused through WRRF, cross-encoder reranked, NLI- and MMR-filtered, and run through the rules engine — heat-weighted, decay-gated, project-scoped. Filter by type= ("memory", "wiki", or both). wiki_query remains for wiki-only search.

Harness task-list mirror

• wiki_write_task_list(project, content, directory) persists the Claude Code harness task list (TaskCreate/TaskList) to the wiki store so it survives /clear and session exit. The stop-hook checkpoint step writes it out; the SessionStart restore-nudge re-injects open tasks. Written through the canonical seam and resolved by directory, so it is reachable from any working tree. A dedicated MCP tool — not a raw wiki_add — because the canonical-write path is structurally bounded to the {project}-task-list slug.

Architecture Decision Records (ADR)

• adr_add(...) writes a per-ADR wiki page (yadgar-adr-NNNN) enforcing an 11-field schema with auto-incrementing IDs. Each ADR gets its own page indexed in a thin yadgar-adr-index; recall() IS the read path (pages are recall-visible and immortal — no decay). adr_get(directory, adr_id) fetches a single ADR; adr_list(directory, status=) reads the index with optional status filter. A stop-hook prompt captures decisions at session end. Yadgar dogfoods its own system with 138 real ADRs.

Agent-prompt library

• Reusable subagent dispatch prompts, stored as tagged wiki pages (agent-prompt-<pattern>), versioned and improving over time. agent_prompt_save(pattern, content, purpose) upserts one page per pattern; agent_dispatch_prelude(pattern) builds a prelude to prepend to a subagent prompt — recall-first contract + mandatory ## Yadgar findings footer. seed_agent_prompts() seeds starter patterns. Lookup is via tagged recall (recall(type="wiki", tags=["agent-prompt"])), kept out of normal recall noise.

Knowledge-graph viz (galaxy)

• yadgar viz serves an interactive 3D galaxy layout of memories, wiki pages, entities, and relationships at http://localhost:42069. The galaxy arranges nodes spatially: loose low-heat memories form the outer halo, recurring semantic clusters spiral as arms, and core/anchor nodes form a central bulge. Heat encodes brightness. The force-directed / 2D engine is removed (ADR-0138) — galaxy is the sole renderer.
• Features: draggable node popups, Fit/Reset galaxy camera, filter by node type, heat slider, hover-neighborhood highlight, search, traces replay (replay a tool trace from Tempo), and an in-browser config panel (System → Config).
• Precomputed server-side layout (unconditional): the nightly/full cycle computes 3D node positions, signature-caches them, and /api/graph serves the precomputed coordinates. On a seed miss, the client runs a cold layout.
• The UI is organized into four menus: Graph · Bookmarks · System {Config, Health, Stats} · Help {Guide, Config Reference, About, Debug}.

Read-only DB inspection (db_inspect)

• db_inspect(query, params, limit) executes a SurrealQL SELECT against a read-only viewer client (YADGAR_RO_PASS), gated by YADGAR_DEBUG_APIS_ENABLED. No writes possible from this surface (ADR-0132). Forwarded by core to the backend /api/debug/read_query endpoint.

Nightly consolidation (the "brain cycle")

Runs nightly while the daemon stays up in maintenance mode (no MCP reconnect). Phases: apply_decay → process_episodes → merge_duplicates → link_similar → detect_causality → memify → cls_consolidation plus a dream/sleep phase.

• CLS promotion — recurring episodic patterns promote to semantic memory (Complementary Learning Systems).
• Causal discovery — PC-algorithm causal-DAG inference over co-occurring memories.
• Dream replay — associative co_occurrence linking + insight generation; insights cap at 21 days if unaccessed.
• Community detection — graph clustering surfaced as clusters in the viz.
• Re-embeds stale memories; precomputes the galaxy graph layout.

Config system

• pydantic-settings with precedence env vars (YADGAR_*) > ~/.config/yadgar/config.yaml > defaults.
• System → Config is an in-browser editor (part of the viz UI): category-grouped knobs, alpha-sorted, hover tooltips, ⓘ deep-links to a Config Reference page, and SOURCE badges distinguishing env-var / config-file / default origins. Writes are bearer-auth gated.

Security & observability

• Bearer-token MCP auth on /api/*, /hooks/*, /mcp — default-deny CORS, timing-safe compare, loopback-only by default. /health and /metrics exempt on loopback.
• Always-on secret gate blocks AWS / GCP / Stripe / Slack / OpenAI / Anthropic keys, JWTs, GitHub PATs, private keys, and DB URIs before they reach the store (cannot be disabled; context-aware allowlist for known-good fixtures).
• Auto-capture sanitization strips ANSI escapes, control chars, and Unicode bidi-override before action-log insert.
• Prometheus /metrics + OpenTelemetry distributed tracing across core + backend (core→backend W3C traceparent joins one trace in Tempo); structured JSON logs; per-phase consolidation duration markers. Tri-signal standard (v5.101, ADR-0034): every in-scope function emits span+metric+log via the @observe decorator, ratcheted by the I33 coverage lint.
• Async write queue (file-queue, ADR-0075) with retry/backoff, dead-letter for permanent failures, and DLQ inspection tools (dlq_inspect, dlq_requeue, dlq_dismiss).

Architecture

┌──────────────┐        MCP (streamable-HTTP / stdio)         ┌──────────────────────────────┐
│  Claude Code │ ◄────────────────────────────────────────── ►│  yadgar/core (:8765)          │
│   + hooks    │  memorize / recall / wiki_* / checkpoint …   │  MCP server (thin router)     │
└──────────────┘                                               │  auth · rules · hooks · viz  │
                                                               └────────────┬─────────────────┘
                                                                            │ HTTP + file queue
                                                               ┌────────────▼─────────────────┐
                                                               │  yadgar/_shared              │
                                                               │  config · storage contracts  │
                                                               │  observability · security    │
                                                               └────────────┬─────────────────┘
                                                                            │ HTTP
                                                               ┌────────────▼─────────────────┐
                                                               │  yadgar/backend               │
                                                               │  SurrealDB store (:8000)      │
                                                               │  embed + rerank (:8001)       │
                                                               │  retrieval pipeline (POST /recall) │
                                                               │  consolidation compute       │
                                                               └──────────────────────────────┘

Three physical layers (ADR-0056 / ADR-0060 / ADR-0062):

• yadgar/core — the MCP server (FastAPI). Routes tool calls; forwards recall to backend via POST /recall; hosts the viz web UI routing layer. No heavy compute lives here.
• yadgar/_shared — contracts, config (pydantic-settings), storage client, observability (@observe), security gate. No direct MCP tool code.
• yadgar/backend — SurrealDB (:8000) + embed/rerank service (:8001). Owns the full retrieval pipeline, consolidation compute, and the async write drainer. ML models are baked into the backend image (ADR-0101).

Write path (ADR-0075): MCP tool → core → file queue (YADGAR_QUEUE_BASE=/data/queue) → backend drainer → SurrealDB. Writes are async by default; wait=True blocks until drainer commits.

Recall path (ADR-0044): recall() in core is a thin forwarder → backend POST /recall runs the full pipeline (FTS + KNN + PPR + spreading activation → WRRF fusion → Ettin-32m CE rerank → NLI → MMR → adversarial → rules). Scoping is by project_id plus the global-reach tag, applied post-fetch; the branch filter and current-branch boost this line used to describe were removed by ADR-0215.

Deeper detail: docs/reference/architecture.md · docs/reference/retrieval.md · docs/reference/memory-lifecycle.md.

Layer docs (in-tree): each layer root carries a README.md (subsystem map) and an AGENTS.md (placement laws for coding agents) — yadgar/_shared/ · yadgar/backend/ · yadgar/core/.

Install

Python 3.14+ on the host (or use the Docker / Compose path for zero host Python). There are three post-install surfaces and they are NOT the same command: yadgar setup (the minimal bootstrap: config, secrets, MCP registration, code_graph), yadgar-setup (the full installer for pipx/brew/nix-profile: everything yadgar setup does plus images, units, hooks, agents, rules, seeds), and make setup (the repo-checkout equivalent of yadgar-setup). All three provision code_graph.

pipx (recommended for isolated install):

# Yadgar needs Python 3.14+ (CPython features used by the package). Stock
# Debian 13 / Ubuntu LTS ship only 3.11/3.12; pin a 3.14 interpreter via
# uv before pipx so the venv it creates targets 3.14.
uv tool install uv                     # one-time: uv itself if missing
uv python install 3.14                # one-time: 3.14 interpreter
pipx install --python "$(uv python find 3.14)" yadgar
yadgar setup

Nix flake:

nix profile install github:m-agahi/yadgar
yadgar setup

Plain pip:

pip install yadgar
yadgar setup

Repo checkout (development):

git clone https://github.com/m-agahi/yadgar.git
cd yadgar
make setup            # install + hooks + agents + units + seed anchors

yadgar setup writes ~/.config/yadgar/config.yaml, generates ~/.config/yadgar/secrets.env (chmod 600) with random YADGAR_MCP_AUTH_TOKEN + SURREAL_PASS + YADGAR_RW_PASS + YADGAR_RO_PASS, registers the MCP server with Claude Code, and provisions code_graph. The hooks, subagent templates, rules, anchor seeds, container images and systemd (Linux) / launchd (macOS) user units come from the full installers — yadgar-setup (pipx/brew/nix-profile) or make setup (repo checkout), which run their own building-block chains rather than calling yadgar setup. All are idempotent — re-run after upgrades.

Background maintenance (v5.169+). The installer now ships the maintenance units on every surface, not just NixOS: a nightly cycle at 19:00 UTC (backup → consolidate → vacuum → backup) and a weekly vacuum on Sunday 04:00 local (±30 min jitter), plus a watcher that services MCP vacuum_now(). Before this, a make setup install on Linux rendered no maintenance units at all — consolidation, heat decay, episode formation and dream replay silently never ran, and on macOS the jobs fired and failed. Both timers are Persistent=true, so a run missed while the machine was off is caught up at next start; the first catch-up on a large never-consolidated DB can take a while (bounded at 1h). Check with systemctl --user list-timers 'yadgar-*', or yadgar-setup --doctor. Mask what you do not want:

systemctl --user mask yadgar-nightly-cycle.timer   # or yadgar-vacuum.timer

These jobs run on the host (the vacuum flow stops and restarts the backend mid-run, which cannot work from inside the container), so they need SurrealDB reachable over HTTP. The backend therefore publishes it on 127.0.0.1:8000, loopback only — a posture change for existing non-nix installs, matching what the nix flake has shipped since v5.46. If port 8000 is already taken, re-point it: make setup YADGAR_BACKEND_SURREAL_PORT=18000. The vacuum will also stop the daemon briefly, so a live MCP session at 04:00 Sunday loses its connection.

systemd lingering (Linux). Yadgar's units are systemd user units, so the installer also enables lingering for your user (loginctl enable-linger) — without it the daemon stops when you log out and never starts at boot. Enabling your own lingering needs no sudo. Note the consequence on a shared host: your yadgar containers keep running (and holding memory) while you are logged out. Opt out with yadgar-setup --no-enable-linger or make setup YADGAR_ENABLE_LINGER=0; uninstalling does not disable lingering again, since it may serve your other user services.

Then start the daemon and register it with Claude Code:

set -a && . ~/.config/yadgar/secrets.env && set +a
yadgar daemon start
yadgar daemon configure-mcp   # writes ~/.claude.json with Authorization: Bearer header

Restart Claude Code, then verify:

yadgar daemon status
yadgar stats

Per-platform detail: docs/reference/install.md.

Stdio-only (no daemon, no Docker)

Single-session use. Skip yadgar setup. Add to ~/.claude.json:

{ "mcpServers": { "yadgar": { "command": "yadgar", "args": [] } } }

Restart Claude Code. No bearer auth; embed / rerank degrade gracefully without the backend. Note: the installed daemon uses streamable-HTTP transport by default; stdio mode is for single-session / no-Docker use.

Docker Compose (recommended for production)

Two containers — backend (SurrealDB + embed/rerank service) and core (MCP server). Independent version tracks.

# Source secrets first
set -a && . ~/.config/yadgar/secrets.env && set +a

docker compose up -d

The compose file (docker-compose.yml) defines:

• yadgar-backend — openfantasy/yadgar-backend:5.55.0, SurrealDB on :8000 + embed/rerank on :8001. Mounts yadgar-db-data (read-only) + yadgar-queue-data (file queue, YADGAR_QUEUE_BASE=/queue-data).
• yadgar-core — openfantasy/yadgar:5.149.0, MCP server on :8765. Mounts yadgar-queue-data at /data (shared file queue). Depends on backend healthcheck before starting.

The knowledge-graph viz UI is not part of the compose service. Run yadgar viz separately on the host to launch the viz server at http://localhost:42069 — it reverse-proxies /api/* to the daemon at :8765.

Then register with Claude Code:

yadgar daemon configure-mcp

Note: for the manual docker run path or systemd-based production deploy, see docs/reference/install.md. The compose file above is the recommended development/production path.

Updating

yadgar update --check   # probes PyPI; prints upgrade command; exit 0

For pipx the suggested command is pipx upgrade yadgar; then re-run yadgar setup (idempotent) to refresh hooks and units. An opt-in update orchestrator (yadgar update --install, default off via update_install_enabled: false) coordinates snapshot → image pull → graceful drain → restart → health-check → CLI upgrade, with automatic rollback (yadgar update --rollback). Operator steps for a breaking change are written to a local, gitignored MIGRATION_NOTES.md at the repo root — it is a hand-off channel, so it is absent from a fresh clone.

Common commands

Setup / install

yadgar setup                                  # first-run: config, secrets, hooks, units
yadgar install --client claude-code --hooks    # (re-)wire Claude Code hooks ONLY — MCP config untouched (replaces yadgar install-hooks)
yadgar install-subagents                      # install subagent templates
yadgar daemon configure-mcp                   # write ~/.claude.json with streamable-HTTP + bearer header
yadgar seed <directory>                       # bootstrap memory for an existing project

Daily use

yadgar daemon start|stop|status|logs|health   # manage the background daemon
yadgar viz                                     # launch knowledge-graph UI at http://localhost:42069
yadgar stats [--project /path]                 # memory counts + health
yadgar daemon restart                          # restart after config change

Maintenance / backup

yadgar vacuum                         # compact the SurrealKV store
yadgar export duckdb --output snap.duckdb    # analytics snapshot (requires yadgar[analytics])
yadgar config set <key> <value>       # change a config knob (also editable in viz System→Config)
yadgar config list                    # show all config knobs with current values
yadgar update --check                 # check for new version on PyPI
yadgar update --install               # orchestrated upgrade (snapshot → pull → restart → verify)

Debugging

yadgar daemon logs                    # tail daemon container logs
yadgar daemon health                  # check /health endpoint
yadgar --version                      # core + backend + daemon versions
yadgar config get YADGAR_LOG_FORMAT   # inspect a single knob
# MCP-level: dlq_inspect() · audit_anchors() · check_invariants() · db_inspect("SELECT ...")

CLI

yadgar                              # MCP server (streamable-HTTP default); --transport {sse,streamable-http}
yadgar daemon start|stop|logs|health|status|restart
yadgar daemon configure-mcp         # write ~/.claude.json with bearer header + streamable-HTTP transport
yadgar daemon install-service       # install systemd / launchd unit
yadgar setup                        # first-run config + secrets + MCP snippet
yadgar stats [--project /path]      # memory statistics
yadgar viz                          # knowledge graph at http://localhost:42069
yadgar vacuum                       # compact the SurrealKV store
yadgar seed <directory>             # bootstrap memory for an existing project
yadgar code-graph index|query|refresh <repo>  # host-side multi-lang code-structure (on by default; MCP config_set code_graph.enabled=false to opt out)
yadgar rules export|import          # policy rules
yadgar config init|list|get|set     # configuration
yadgar update --check|--install|--rollback
yadgar export duckdb --output snap.duckdb   # analytics snapshot (pip install yadgar[analytics])
yadgar install --client <name> [--hooks] [--scope ...]   # no surface flag = MCP + rules + hooks; --hooks = hooks surface only
yadgar install-subagents                # install subagent templates (claude-code only)
yadgar restore <directory>          # (hook-internal) restore context post-compaction
yadgar capture                      # (hook-internal) lightweight action capture
yadgar drain                        # (hook-internal) flush file queue
yadgar context                      # (hook-internal) session-start context inject

MCP tools

Yadgar exposes 87 MCP tools (89 @_tool decorators minus 2 test-only stubs; counted 2026-08-28). ⚡ marks power=True tools (gated in the minimal MCP profile). A categorized selection — full list via the running server's tool catalog:

A typed JavaScript / TypeScript SDK wraps the tool surface as async methods (Node.js, Vercel Edge, Cloudflare Workers, Deno) — see docs/reference/sdk-js.md.

Configuration

yadgar config init        # write ~/.config/yadgar/config.yaml
yadgar config set retrieval_profile fast

Priority: env vars (YADGAR_*) > ~/.config/yadgar/config.yaml > defaults. Knobs are also editable in-browser via System → Config in the viz UI. Key vars (full reference in docs/reference/configuration.md):

Subagent integration

Yadgar ships a SubagentStop hook that captures memory findings from Claude Code subagents: when a subagent completes, its final report is scanned for a ## Yadgar findings section and each bullet is persisted with provenance_agent set to the agent type.

To opt your subagents into the protocol, paste docs/reference/claude-subagent-contract.md into your ~/.claude/CLAUDE.md, then run yadgar install --client claude-code --hooks --scope global. The contract is opt-in — Yadgar works without it.

The agent-prompt library (agent_prompt_save / agent_dispatch_prelude) gives subagent dispatch prompts a versioned home: save a good prompt once, improve it over time, and every dispatch pulls the latest version automatically. agent_dispatch_prelude(pattern, task_topic) builds a compliant prelude (recall-first contract + ## Yadgar findings footer) from the stored pattern.

Benchmark

Yadgar is evaluated on LongMemEval (ICLR 2025), the standard academic benchmark for long-term conversational memory. The longmemeval_s variant runs 500 questions across 6 categories against ~50 sessions of synthetic history per query, scored on Phase 1 retrieval (does the memory layer surface gold-context sessions in top-k?) and Phase 2 QA accuracy (does the reader produce the gold answer, judged by an LLM grader?).

Headline (v5.26.0, full 500q, Sonnet 4.6 reader + judge — the most recent full run; subsequent versions targeted infrastructure, viz, wiki, and recall-unification rather than retrieval quality):

Yadgar beats Zep by 5.6 pp on the same 500-question sample. mem0 leads by 25 pp via LLM-extract-on-ingest. Phase 1 retrieval: MRR 0.928, Recall@10 0.906, NDCG@10 0.863 — the memory layer surfaces gold context for ~91% of queries, so the remaining QA gap is mostly reader synthesis, not retrieval.

Full methodology and per-type breakdown: docs/benchmark-results/BENCHMARK_RESULTS.md.

Recall speed

Ettin-32m @ --cpus 3 (ADR-0106 standing config), warm steady-state p50 ~2.6s / mean ~2.3s (n=30, controlled 2026-07-15).

The cross-encoder reranker swap (GTE-ModernBERT → Ettin-32m, v5.132.0/5.43.0) delivered a measured 2.44× end-to-end recall speedup on equal hardware (same-image, same-CPU A/B, histogram-delta method per ADR-0098). The 3-CPU standing config (ADR-0106) adds parallel batch scoring via gather_budget=2 (~24% further reduction vs 2-CPU). CE is ~25% of the cold recall wall (ADR-0105) — the dominant gains come from the model swap.

Controlled re-measurement 2026-07-15 (v5.143.0/5.50.0, n=30 per regime, CE-miss gate PASS): warm steady-state mean 2,332ms / p50 2,644ms / p95 3,064ms; cold (post-restart) mean 2,594ms / p50 2,682ms / p95 3,148ms.

Full consolidated latency history, measurement protocol, comparability caveats, and per-milestone verdicts: docs/benchmark-results/RECALL_SPEED.md.

Roadmap

Shipped highlights (v5.78 → current)

• Unified recall (v5.78–v5.81) — memory + wiki merged into one ranked, heat-weighted, project-scoped result; now the default.
• ADR tooling (adr_add / adr_get / adr_list, v5.85+) + capture-first Stop-hook prompt. Per-ADR wiki pages with thin index; recall() is the read path. 138 real ADRs dogfooded.
• Agent-prompt library (v5.85, ADR-0007) — wiki-backed, tagged-recall lookup; agent_dispatch_prelude builds compliant subagent preludes.
• Harness task-list mirror — wiki_write_task_list (stop-hook out) + session-start restore-nudge (in); persists the Claude Code task list across /clear.
• Galaxy viz (post-#52, ADR-0134/0138) — galaxy is the sole renderer (force-directed/2D engine removed); galaxy layout (loose/core/arms), traces replay, in-browser config panel.
• Read-only DB inspection (db_inspect, ADR-0132) — gated SurrealQL SELECT surface.
• Wiki autolink + repo-wiki store-bridge (v5.85).
• Viz overhaul + precomputed server-side layout + System → Config editor (v5.86–v5.88).
• Core/_shared/backend layer split (ADR-0056/0060/0062/0063) — import-linter-enforced three-layer architecture; forward-only recall (ADR-0044) with full pipeline in backend.
• Recall perf + accounting (v5.97–v5.104) — WRRF N+1 batches, spreading-activation N+1 batched; CE is ~25% of cold recall wall (ADR-0105 corrected).
• Ettin-32m CE reranker (Train 4, ADR-0104) — 2.44× end-to-end recall speedup; 3-CPU standing config (ADR-0106).
• Tri-signal observability standard (v5.100–v5.101, ADR-0034) — span+metric+log per function, I33 coverage lint, core→backend W3C traceparent, OTLP → Tempo.
• Test-speed train (v5.104, ADR-0036) — CI shards ~2× faster.
• File-queue write path (ADR-0075) — async write queue on shared volume; both containers share yadgar-queue-data.
• Deps modernization (ADR-0100) — transformers 5.x; optimum-onnx removed.
• Module standardization (ADR-0128/0130) — major subsystems promoted to package dirs; import-linter enforced; perf-neutral (ADR-0129).

Open horizon

• v6 — Nightly LLM curator. A local agent (Ollama; two-tier deepseek-r1 + qwen3:8b) runs each night to detect staleness, annotate contradictions, find semantic correlations beyond co-occurrence, propose merges/forgets, and dedupe wiki pages.
• v7 — Real-time synthesis. recall(synthesize=True) / ask() tool; depends on a sub-10 s local synthesis model.
• Open work: improvement-train A1+C4 · full-observability per-area rollout (#I33) · ci-velocity remaining (#83/#79) · cpu-burst Part 2 · task-routing fix · fusion tiebreak determinism · obs velocity completion.

Full history: CHANGELOG.md.

Documentation

• AGENTS.md — operational guide for AI coding agents (setup, dev env, tests, code style, PR rules)
• Architecture — component map, retrieval, security, observability
• Memory lifecycle — heat, archiving, pruning, consolidation phases
• Retrieval — fusion, rerank, project scoping, pipeline stages
• Configuration — configuration reference (env vars, precedence, key knobs)
• Install — per-platform setup + the yadgar setup --doctor probe
• JS/TS SDK — typed client for the MCP tool surface
• Release runbook · Migration notes (local, gitignored MIGRATION_NOTES.md) · Subagent contract

Contributing

Every change to yadgar/** must update README.md and docs/ in the same PR. Conventional Commits format. No Co-Authored-By: trailers. AI coding agents working on this repo: read AGENTS.md for the full dev / test / PR contract.

Related projects

• ccpm — Claude Code Plugin Marketplace. Ships code-review, confluence-rfc, git-flow, repo-wiki, tf-naming-check, and update-jira plugins that compose with yadgar.

Tribute

Inspired by Zikkaron by @amanhij. Different architecture, same north star.

Yadgar stands on a great deal of open-source work — models, databases, frameworks, and tools. Full credits and acknowledgments (including Tom Aarsen, whose sentence-transformers + Ettin reranker underpin our retrieval): docs/reference/tributes.md.

License

Apache 2.0. See LICENSE.

Third-party licenses (key dependencies)

• SurrealDB (default storage backend) — Business Source License 1.1 (BSL). The Additional Use Grant permits embedded use; yadgar bundles + operates SurrealDB as a single-tenant per-deployment store, which falls under that grant. Non-compliant trigger: offering a hosted multi-tenant managed yadgar service exposing the SurrealDB API directly to third-party customers — get a commercial SurrealDB license or migrate to Postgres + pgvector if that becomes a goal.
• surrealdb Python SDK — Apache 2.0 (separately licensed from the server).
• embed / rerank models — sentence-transformers/all-MiniLM-L6-v2, cross-encoder/ettin-reranker-32m-v1 (CE primary), cross-encoder/ettin-reranker-68m-v1 (fallback), Alibaba-NLP/gte-reranker-modernbert-base (rollback), cross-encoder/ms-marco-MiniLM-L-6-v2, cross-encoder/nli-deberta-v3-small: Apache 2.0 weights via Hugging Face.
• Benchmarks (benchmarks/): scripts Apache 2.0, but datasets carry their own licenses — LoCoMo is CC BY-NC 4.0 (non-commercial only); LongMemEval MIT. See benchmarks/README.md.

Full per-dependency audit: docs/reports/audits/license-compliance-audit-2026-05-30.md.