archiver-rag 0.2.0

A finding aid for your knowledge graph

The agent-agnostic memory management system for your Obsidian vault

Beta. archiver-rag is under active development and I'm looking for feedback. Tools and config may change before 1.0; see the changelog. Bugs, ideas, rough edges: open an issue.

Archiver RAG turns your Obsidian vault into a live, queryable knowledge graph that any MCP-compatible AI agent can search, update, and reorganize — without ever leaving its native interface.

Connect it once. Every agent you use (Claude Code, Cursor, Gemini CLI, or your own) gets semantic search, automatic knowledge logging, wikilink-aware graph traversal, and vault health monitoring out of the box.

---

How it works

Your Obsidian vault (.md files)
         ↓  file watcher + ingest pipeline
     ChromaDB  (persistent vector store)
         ↓  MCP server
     Any MCP-compatible agent

Three layers make search smarter than plain embeddings:

1. Contextual prefix — each chunk is embedded with its note's metadata (folder, tags, wikilinks), so vectors carry structural context
2. Rich metadata filtering — ChromaDB stores folder, type, tags, incoming link count, and wikilinks for filtered retrieval
3. Graph reranking — after vector search, results are re-scored by wikilink proximity to a context note and hub importance

The file watcher runs as a background service. Edit a note in Obsidian, save it, and it's indexed and auto-linked within seconds — no manual sync needed.

---

Features

• Semantic search with graph reranking — finds notes by meaning, then boosts results connected via wikilinks
• Adaptive auto-linking — after every ingest, the ## Related section is rebuilt with [[wikilinks]] to every note that clears a relevance margin of the top match — the list grows and shrinks with the graph instead of accumulating
• Knowledge logging — create categorized notes (decision, lesson, gotcha, pattern, …) from any agent; date lives in frontmatter, filename is the slug identity wikilinks are written against
• Semantic folder placement — folders declare a description (_folder.md, generated by archiver-rag describe or auto-regenerated by the watcher); new notes are scored against those descriptions and moved to the best match above a threshold, falling back to the note's frontmatter type: field
• Soft delete — archiver-rag delete moves notes to .trash/ (Obsidian's delete convention, recoverable) and sweeps inbound [[wikilinks]] from the rest of the vault
• Vault health — single call returns orphaned notes, broken links, missing frontmatter, tag stats, and recent activity
• Wikilink-aware reorganization — move files and every [[link]] across the vault is rewritten automatically
• Smart clustering — label-propagation algorithm groups notes by wikilink structure and suggests folder organization (manual command — the watcher runs semantic placement, not full-graph clustering)
• Inbox routing (opt-in) — when no folder claims a note, it can be parked in inbox/ and a new real folder is spun out automatically once a group of embedding-similar notes accumulates there (off by default)
• Agent-agnostic — exposes a standard MCP interface over stdio or HTTP; works with any MCP-compatible client

---

Requirements

• Python >= 3.11
• pipx or uv for installation
• An Obsidian vault (local .md files)
• An MCP-compatible agent (Claude Code, Cursor, etc.)

---

Installation

With pipx:

pipx install archiver-rag

Or with uv:

uv tool install archiver-rag

If your system Python is older than 3.11, uv can fetch a suitable one: uv tool install --python 3.12 archiver-rag (pipx: pipx install --python python3.12 archiver-rag, with that Python already installed).

Use pipx or uv tool, not pip install — both create an isolated environment and expose the CLI globally on PATH, which is required for MCP registration to find the correct executable.

For local development from a clone of this repo, use pipx install --editable . instead.

---

Development

git clone https://github.com/FernandoJRR/archiver-rag && cd archiver-rag
pipx install --editable .   # global CLI — required for MCP registration
pip install --group dev -e .  # adds pytest, ruff
pytest                      # 447 tests, ~5 s

Tests marked slow load the sentence-transformers model; skip them with -m "not slow".

tests/ layout (highlights — see AGENTS.md for the full file-by-file list):

• conftest.py — _no_real_vault (autouse): patches get_vault_path in every module that imports it, so no test ever touches the real vault. _no_real_home_paths (autouse): redirects archiver_rag.paths' config/data/cache dirs so no test ever touches the real ~/.config/archiver-rag/, ~/.local/share/archiver-rag/, or ~/.cache/archiver-rag/. Opt-in tmp_vault and tmp_install fixtures for tests that need real files.
• test_wikilinks.py — 28 unit tests for the offset-based wikilink extractor
• test_linker_section.py — 11 characterization tests for _append_links_section

---

Setup

Run the one-time setup wizard:

archiver-rag init

This will:

1. Ask for your vault path
2. Index your vault into ChromaDB
3. Register the MCP server in ~/.claude.json (or prompt you to do it manually for other clients)
4. Install the background watcher as a launchd agent (Mac) or systemd service (Linux)

---

MCP registration (manual)

If you prefer to register manually, add this to your MCP client config:

{
  "mcpServers": {
    "archiver-rag": {
      "command": "/path/to/archiver-rag",
      "args": ["serve"]
    }
  }
}

Find the executable path with which archiver-rag.

For Claude Code specifically, use:

claude mcp add --scope user archiver-rag $(which archiver-rag) serve

---

HTTP transport

By default the server speaks MCP over stdio: your client spawns it as a child process. That means one client per server, and no way to reach the vault from another machine. --transport http serves the same six tools over streamable HTTP instead, so several clients can share one warm process:

archiver-rag serve --transport http            # foreground, http://127.0.0.1:8077/mcp

Or run it as a supervised background daemon (launchd on Mac, systemd on Linux) instead of holding a terminal:

archiver-rag start http          # installs + starts the daemon (asks about auto-start at login)
archiver-rag stop http           # stop it (settings are kept)
archiver-rag restart http        # restart
archiver-rag status              # includes the HTTP daemon's state and URL

start http refuses to start if the port is already bound, so it can never silently crash-loop behind a busy foreground serve.

Register it with Claude Code:

claude mcp add --scope user --transport http archiver-rag http://127.0.0.1:8077/mcp

Other clients

Both transports work with any MCP-compatible client. Replace /path/to/archiver-rag with the output of which archiver-rag.

opencode

Add an mcp block to ~/.config/opencode/opencode.jsonc (or opencode.json, or a project-local file of either name — opencode's schema allows comments and trailing commas in both):

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    // stdio — opencode launches the server itself
    "archiver-rag": {
      "type": "local",
      "command": ["/path/to/archiver-rag", "serve"],
      "enabled": true,
      "timeout": 30000
    },
    // HTTP — connects to a server you started separately
    "archiver-rag-http": {
      "type": "remote",
      "url": "http://127.0.0.1:8077/mcp",
      "enabled": true,
      "timeout": 30000
    }
  }
}

Use one or the other — both are shown here only to give the shape of each.

Raise timeout. opencode defaults to 5000 ms per request, and the first search_vault call loads the embedding model — measured at ~5 s on a warm disk, right at the limit. Without a higher timeout the first search may fail and then succeed on a retry, which is a confusing way to meet the tool. Later calls take ~70 ms.

Codex CLI

# stdio — Codex launches the server itself
codex mcp add archiver-rag -- $(which archiver-rag) serve

# HTTP — connects to a server you started separately
codex mcp add archiver-rag-http --url http://127.0.0.1:8077/mcp

Both write to ~/.codex/config.toml, and you can equally hand-edit it:

[mcp_servers.archiver-rag]
command = "/path/to/archiver-rag"
args = ["serve"]

[mcp_servers.archiver-rag-http]
url = "http://127.0.0.1:8077/mcp"

Verify with codex mcp list. If your server sits behind a proxy that wants a token, --bearer-token-env-var NAME reads it from the environment — archiver-rag itself never checks it (see below).

Reaching it from another machine

archiver-rag performs no authentication and terminates no TLS. Anyone who can reach the port has full access: the entire vault is readable, and log_note and move_notes can modify it.

It is deliberately not this tool's job to decide how you secure that. Keep the server on loopback and put a layer you already trust in front of it — a reverse proxy terminating TLS, an SSH tunnel, a VPN, or a private overlay network. The server does not need to know which; it stays on plain HTTP at 127.0.0.1:8077 in every case.

If you bind beyond loopback (--host 0.0.0.0), the CLI prints a warning — heed it. You can additionally enable DNS-rebinding protection by naming the hostnames you expect to serve:

archiver-rag serve --transport http --allowed-host vault.internal.example

Requests arriving with any other Host header are rejected with 421. The bind address itself is always allowed, so local access keeps working.

Options

Flag	Default	Meaning
--transport	stdio	stdio or http
--host	127.0.0.1	Bind address
--port	8077	Bind port
--path	/mcp	HTTP route
--allowed-host	(none)	Enable DNS-rebinding protection for this Host (repeatable)
--stateful	off	Use HTTP sessions + SSE instead of stateless JSON

http_host, http_port and http_path in config.json supply defaults for the corresponding flags.

---

Agent instructions (skills)

Registering the MCP server gives an agent access to the tools — but agents tend to fall back on their own internal memory instead of reaching for the vault. The instruction files in skill/ fix that: they enforce a vault-first rule so the agent searches and stores knowledge in your vault before anything else.

What the skill enforces:

• Before answering or reading source files — call search_vault first; only fall back to internal memory if the vault returns nothing relevant
• When something important is missing from the vault — proactively log_note it. If a fact, decision, or piece of context matters to the overall picture and a search_vault came back empty, record it so the knowledge graph grows instead of letting that context die in a single session
• After solving a non-trivial problem — call log_note to capture the decision/lesson/gotcha back into the vault
• The vault is the authoritative memory system — internal agent memory is a fallback only

A version is provided for each agent, since each loads instructions differently:

Agent	File	Install to
Claude Code	skill/claude-code/SKILL.md	~/.claude/skills/archiver-rag/SKILL.md (on-demand skill)
OpenCode	skill/opencode/AGENTS.md	project root AGENTS.md or ~/.config/opencode/AGENTS.md
Codex CLI	skill/codex/AGENTS.md	project root AGENTS.md or ~/.codex/AGENTS.md
GitHub Copilot	skill/copilot/copilot-instructions.md	.github/copilot-instructions.md

Each file is self-contained — it includes the MCP registration snippet for that agent plus the full vault-first rules and tool reference. For Claude Code the file is an on-demand skill; for the others it's an always-on instruction file (loaded into every session), which makes the vault-first behavior unconditional.

---

CLI reference

archiver-rag init                      # one-time setup wizard
archiver-rag start                     # start the background watcher service (default target)
archiver-rag start http                # install + start the detached MCP HTTP server (launchd/systemd)
archiver-rag stop [watcher|http]       # stop the watcher or the HTTP server
archiver-rag restart [watcher|http]    # restart the watcher or the HTTP server
archiver-rag status                    # service liveness, watcher activity, index drift, config
archiver-rag status --json             # same report as JSON
archiver-rag index                     # force re-index the entire vault (runs prune_orphans)
archiver-rag sync                      # ingest only new/modified notes + prune orphaned chunks
archiver-rag prune                     # remove index chunks whose source file no longer exists
archiver-rag search "query"            # test semantic search from the terminal
archiver-rag health                    # index-vs-disk drift + vault health (orphans, broken links)
archiver-rag health --json             # same report as JSON
archiver-rag logs                      # tail the service log

# Knowledge logging
archiver-rag log "Title" --type decision --tag arch --related NoteA
archiver-rag delete <note>... [--yes]  # move to .trash/ + sweep inbound wikilinks (recoverable)

# Folder descriptions (drive semantic placement)
archiver-rag describe                  # generate missing _folder.md descriptions
archiver-rag describe --all            # regenerate all source:auto descriptions
archiver-rag describe --folder <folder> [--set "term1 term2"]   # one folder; --set marks source:manual

# Placement & clustering
archiver-rag place <note>              # suggest folder (semantic + type fallback)
archiver-rag place <note> --apply      # move the note immediately
archiver-rag place --all               # dry-run: current vs suggested folder for every note
archiver-rag place --all --apply       # batch move all notes to their suggestion
archiver-rag cluster [--min-size 2] [--apply]   # [EXPERIMENTAL] label propagation over the wikilink graph (manual/diagnostic only)
archiver-rag relink                    # dry-run: report ## Related before/after under the margin rule
archiver-rag relink --apply            # one-time repair: rebuild every note's ## Related section

# Config
archiver-rag config --auto-cluster / --no-auto-cluster        # watcher auto-placement on/off
archiver-rag config --cluster-threshold 5                     # vestigial — kept for old configs
archiver-rag config --placement-threshold 0.55                # cosine threshold for semantic placement
archiver-rag config --type-fallback / --no-type-fallback      # fall back to frontmatter type: field
archiver-rag config --auto-describe / --no-auto-describe      # watcher regenerates _folder.md on membership change

archiver-rag uninstall         # remove all data, both services, and MCP registration

start http takes the same transport flags as serve --transport http (--host, --port, --path, --stateful, --allowed-host), bakes them into the installed service, and asks at runtime whether to start automatically at login (--login / --no-login to script it). It refuses to start if the port is already bound. stop http / restart http manage the same service; uninstall removes both services.

---

MCP tools (for agents)

Once registered, agents have access to 6 tools:

Tool	What it does
search_vault	Semantic search with graph reranking. context_note boosts wikilink neighbors. type filters by frontmatter type: (stable across folder moves). tags post-filters by tag overlap.
vault_status	Vault structure, health diagnostics, tag stats, and recent activity in one call.
get_connections	BFS wikilink traversal — outgoing and incoming links up to depth 3.
move_notes	Move files and auto-rewrite all [[wikilinks]] across the vault.
log_note	Create a knowledge note at {type}/{slug}.md; watcher indexes and auto-links it immediately.
suggest_folder	Suggestion only — never moves anything. Suggests a folder for one note by semantic similarity against declared folder descriptions (_folder.md), matching the same config the CLI place command and the watcher use; falls back to the note's frontmatter type:. Returns reason (semantic / type / none), per-folder scores, and a secondary neighbor_vote. Use move_notes to act on the result.

Whole-vault label-propagation clustering (formerly the cluster_vault MCP tool) is manual/diagnostic-only now — see the experimental archiver-rag cluster CLI command above.

---

Configuration

All runtime config lives at the XDG config path (~/.config/archiver-rag/config.json on Linux/macOS, resolved by archiver_rag/paths.py):

{
  "vault_path": "/path/to/your/vault",
  "install_path": "/Users/you/.local/share/archiver-rag",
  "chroma_path": "/Users/you/.local/share/archiver-rag/chroma_db",
  "auto_cluster": false,
  "cluster_threshold": 5,
  "placement_similarity_threshold": 0.55,
  "type_fallback": true,
  "auto_describe": false,
  "auto_inbox": false,
  "http_host": "127.0.0.1",
  "http_port": 8077,
  "http_path": "/mcp",
  "advanced": {
    "term_extraction_min_notes": 4,
    "alpha_curve": {"type": "log", "scale": 1.0},
    "mmr_lambda": 0.5,
    "max_terms": 6,
    "tag_terms_in_description": true,
    "placement_weights": {"identity": 0.6, "content": 0.4},
    "name_prefix_bonus": 0.15,
    "folder_vacancy_grace_periods": 3,
    "link_margin": 0.05,
    "max_total_links": 15,
    "inbox_min_cluster_size": 3,
    "inbox_similarity_threshold": 0.5
  }
}

Data (the ChromaDB index and centroids.json, the per-folder description centroid cache) lives at ~/.local/share/archiver-rag/, and the cache dir at ~/.cache/archiver-rag/ holds runtime.json — the watcher's heartbeat that archiver-rag status reads to report activity, counters, and crash loops. An existing pre-XDG ~/.archiver-rag/ install is migrated automatically and non-destructively the first time any archiver-rag command runs — the old directory is left in place, never deleted.

Key by key:

• auto_cluster — the watcher runs semantic placement (suggest_folder) for new notes and moves the file to the best-matching described folder. It never runs full-graph label-propagation clustering automatically — that stays a manual, experimental archiver-rag cluster command.
• cluster_threshold — vestigial; kept so old configs still load. It only mattered for the whole-vault clustering fallback the watcher used to run.
• placement_similarity_threshold — cosine similarity a folder must clear to win semantic placement (0–1).
• type_fallback — when no folder clears the threshold, place the note in its frontmatter type: folder instead.
• auto_describe — the watcher regenerates a folder's _folder.md description (blended adaptively, never touching source: manual) whenever a note is created, deleted, or moved in or out of it.
• auto_inbox — route notes no folder claims into inbox/, and spin a new described folder out of it once inbox_min_cluster_size embedding-similar notes accumulate. Off by default.
• http_host / http_port / http_path — defaults for the HTTP transport, shared by serve --transport http and start http.
• advanced — tuning knobs: description extraction (term_extraction_min_notes, alpha_curve, mmr_lambda, max_terms, tag_terms_in_description), semantic placement (placement_weights, name_prefix_bonus), folder lifecycle (folder_vacancy_grace_periods — how long an emptied source: auto folder may stay undescribed before its _folder.md is archived to .archive/), auto-linking (link_margin, max_total_links), and inbox clustering (inbox_min_cluster_size, inbox_similarity_threshold).

---

The knowledge graph model

The vault is treated as a knowledge graph, not a file hierarchy. Notes are nodes; wikilinks are edges. Relationships range from tight (direct links) to loose (semantic proximity surfaced by search).

Note types are expressed through frontmatter, not folder structure:

---
type: decision
tags: [architecture, async]
related:
  - AsyncLocalStorage
  - PrismaExtensions
date: 2026-04-27
---

The ## Related section at the bottom of each note is managed automatically by the auto-linker after every ingest. Don't edit it manually — it will be overwritten.

---

Roadmap

Features on the way:

• RAG-Anything integration — extend ingestion beyond Markdown to handle PDFs, Office documents, images, and other file types, so the vault can become a true multi-format knowledge base rather than .md-only.
• Archiver subagents — dedicated subagents that take over vault management (search, logging, reorganization, clustering) on the main agent's behalf, so the primary agent can delegate knowledge work instead of context-switching into it.

---

License

MIT