archiver-rag 0.2.1

A finding aid for your knowledge graph

The agent-agnostic memory management system for your Obsidian vault

Beta. archiver-rag is under active development and I'm looking for feedback. Tools and config may change before 1.0; see the changelog. Bugs, ideas, rough edges: open an issue.

Archiver RAG is a self-hosted, self-organizing memory system for your AI agents, built on your Obsidian vault. Any MCP-compatible agent can search it, add to it and reorganize it, without leaving its own interface.

Connect it once. Every MCP-compatible agent you use (Claude Code, Claude Desktop, OpenCode, Codex, or your own) gets semantic search, automatic knowledge logging, wikilink-aware graph traversal, and vault health monitoring out of the box.

Why archiver-rag

A memory system your agents share, and one you can shape yourself.

• Self-hosted. Memory, index and embedding model all run on your own machine. No cloud service to depend on, no embedding API to pay for, no one else holding what your agents know.
• One memory system, every agent. Claude Code, Claude Desktop, Codex and OpenCode all read from and write to the same vault, so what one agent learns, the next one already knows.
• You can change it too. It's your Obsidian vault: write down your own ideas, correct what's wrong, delete what's stale, and every agent works from that from then on.
• A memory system that looks after itself. The experiment is that notes are linked, described and filed by meaning as they're saved, with no one, human or agent, organizing it by hand. The goal is a vault that knows what's inside it and keeps itself in order. Linking is always on; automatic filing, folder descriptions and the inbox are off by default while the experiment matures (see What's on after setup).
• No lock-in. Stop using archiver-rag and you still have a normal Obsidian vault in plain Markdown.

Not for you if you want an agent to quietly remember everything from every conversation. archiver-rag keeps what you or your agents decide is worth writing down.

How it works

Your Obsidian vault (.md files)
         ↓  file watcher + ingest pipeline
     ChromaDB  (persistent vector store)
         ↓  MCP server
     Any MCP-compatible agent

Search combines three layers, designed to add structural signal on top of plain embeddings:

1. Contextual prefix — each chunk is embedded with its note's metadata (folder, tags, wikilinks), so vectors carry structural context
2. Rich metadata filtering — ChromaDB stores folder, type, tags, incoming link count, and wikilinks for filtered retrieval
3. Graph reranking — after vector search, results are re-scored by wikilink proximity to a context note and hub importance

Graph reranking needs links to work with: on a new or sparsely linked vault it adds little, and it contributes more as auto-linking fills in each note's ## Related section.

The file watcher runs as a background service. Edit a note in Obsidian, save it, and it's indexed and auto-linked within seconds — no manual sync needed.

Features

• Semantic search with graph reranking — finds notes by meaning, then boosts results connected via wikilinks
• Adaptive auto-linking — after every ingest, the ## Related section is rebuilt with [[wikilinks]] to every note that clears a relevance margin of the top match — the list grows and shrinks with the graph instead of accumulating
• Knowledge logging — create categorized notes (decision, lesson, gotcha, pattern, …) from any agent; date lives in frontmatter, filename is the slug identity wikilinks are written against
• Semantic folder placement — folders declare a description (_folder.md, generated by archiver-rag describe, or kept up to date by the watcher with auto_describe); notes are scored against those descriptions and moved to the best match above a threshold, falling back to the note's frontmatter type: field. Run it on demand with archiver-rag place, or turn on auto_cluster to place every new note automatically (off by default)
• Soft delete — archiver-rag delete moves notes to .trash/ (Obsidian's delete convention, recoverable) and sweeps inbound [[wikilinks]] from the rest of the vault
• Vault health — single call returns orphaned notes, broken links, missing frontmatter, tag stats, and recent activity
• Wikilink-aware reorganization — move files and every [[link]] across the vault is rewritten automatically
• Inbox routing (opt-in) — when no folder claims a note, it can be parked in inbox/ and a new real folder is spun out automatically once a group of embedding-similar notes accumulates there (off by default; needs auto_cluster on too)
• Agent-agnostic — exposes a standard MCP interface over stdio or HTTP; works with any MCP-compatible client
• Whole-vault clustering (experimental) — archiver-rag cluster groups notes by wikilink structure and suggests folders. Preview first; --apply asks before moving anything. Semantic placement is the recommended way to organize.

Requirements

• macOS or Linux (Windows is not supported yet)
• Python >= 3.11
• pipx or uv for installation
• An Obsidian vault (local .md files)
• An MCP-compatible agent, such as Claude Code, Claude Desktop, OpenCode or Codex

Installation

With pipx:

pipx install archiver-rag

Or with uv:

uv tool install archiver-rag

If your system Python is older than 3.11, uv can fetch a suitable one: uv tool install --python 3.12 archiver-rag (pipx: pipx install --python python3.12 archiver-rag, with that Python already installed).

Use pipx or uv tool, not pip install — both create an isolated environment and expose the CLI globally on PATH, which is required for MCP registration to find the correct executable.

For local development from a clone of this repo, use pipx install --editable . instead.

Install size

The install is large because of PyTorch, which runs the embedding model: about 1.4 GB. The first run also downloads the embedding model (all-MiniLM-L6-v2, about 90 MB) once.

On Linux, pip installs the CUDA build of PyTorch by default, which adds several GB you don't need, since archiver-rag runs on the CPU. For a CPU-only install:

pipx install archiver-rag --pip-args="--extra-index-url https://download.pytorch.org/whl/cpu"
# or
uv tool install archiver-rag --index https://download.pytorch.org/whl/cpu

Development

See CONTRIBUTING.md for the full setup, test markers, commit style and release steps.

git clone https://github.com/FernandoJRR/archiver-rag && cd archiver-rag
pipx install --editable .   # global CLI — required for MCP registration
pip install --group dev -e .  # adds pytest, ruff
pytest                      # 451 tests, ~5 s

Tests marked slow load the sentence-transformers model; skip them with -m "not slow".

tests/ layout (highlights — see AGENTS.md for the full file-by-file list):

• conftest.py — _no_real_vault (autouse): patches get_vault_path in every module that imports it, so no test ever touches the real vault. _no_real_home_paths (autouse): redirects archiver_rag.paths' config/data/cache dirs so no test ever touches the real ~/.config/archiver-rag/, ~/.local/share/archiver-rag/, or ~/.cache/archiver-rag/. Opt-in tmp_vault and tmp_install fixtures for tests that need real files.
• test_wikilinks.py — 28 unit tests for the offset-based wikilink extractor
• test_linker_section.py — 11 characterization tests for _append_links_section

Setup

Run the one-time setup wizard:

archiver-rag init

This will:

1. Ask for your vault path, and whether to start the watcher automatically at login
2. Index your vault into ChromaDB
3. Register the MCP server for Claude Code (~/.claude.json); for other clients see Manual registration
4. Install the background watcher as a launchd agent (Mac) or systemd service (Linux), if you chose to

What's on after setup

Nothing in your vault moves unless you turn these on. To enable the self-organizing features:

archiver-rag describe --all                     # give your folders descriptions to place notes against
archiver-rag config --auto-cluster --auto-describe

auto_inbox has no config flag yet: set "auto_inbox": true in config.json. It only works with auto_cluster on. All three take effect on the next note you save, no restart needed.

Manual registration

For clients other than Claude Code, or if you skipped init, add this to your MCP client config:

{
  "mcpServers": {
    "archiver-rag": {
      "command": "/path/to/archiver-rag",
      "args": ["serve"]
    }
  }
}

Find the executable path with which archiver-rag.

For Claude Code, if you didn't use init:

claude mcp add --scope user archiver-rag $(which archiver-rag) serve

If init already registered it, this reports that archiver-rag already exists and changes nothing.

HTTP transport

By default the server speaks MCP over stdio: your client spawns it as a child process. That means one client per server, and no way to reach the vault from another machine. --transport http serves the same six tools over streamable HTTP instead, so several clients can share one warm process:

archiver-rag serve --transport http            # foreground, http://127.0.0.1:8077/mcp

Or run it as a supervised background daemon (launchd on Mac, systemd on Linux) instead of holding a terminal:

archiver-rag start http          # installs + starts the daemon (asks about auto-start at login)
archiver-rag stop http           # stop it (settings are kept)
archiver-rag restart http        # restart
archiver-rag status              # includes the HTTP daemon's state and URL

start http refuses to start if the port is already bound, so it can never silently crash-loop behind a busy foreground serve.

Register it with Claude Code:

claude mcp add --scope user --transport http archiver-rag http://127.0.0.1:8077/mcp

Other clients

Both transports work with any MCP-compatible client. Replace /path/to/archiver-rag with the output of which archiver-rag.

opencode

Add an mcp block to ~/.config/opencode/opencode.jsonc (or opencode.json, or a project-local file of either name — opencode's schema allows comments and trailing commas in both):

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    // stdio — opencode launches the server itself
    "archiver-rag": {
      "type": "local",
      "command": ["/path/to/archiver-rag", "serve"],
      "enabled": true,
      "timeout": 30000
    },
    // HTTP — connects to a server you started separately
    "archiver-rag-http": {
      "type": "remote",
      "url": "http://127.0.0.1:8077/mcp",
      "enabled": true,
      "timeout": 30000
    }
  }
}

Use one or the other — both are shown here only to give the shape of each.

Raise timeout. opencode defaults to 5000 ms per request, and the first search_vault call loads the embedding model — measured at ~5 s on a warm disk, right at the limit. Without a higher timeout the first search may fail and then succeed on a retry, which is a confusing way to meet the tool. Later calls take ~70 ms.

Codex CLI

# stdio — Codex launches the server itself
codex mcp add archiver-rag -- $(which archiver-rag) serve

# HTTP — connects to a server you started separately
codex mcp add archiver-rag-http --url http://127.0.0.1:8077/mcp

Both write to ~/.codex/config.toml, and you can equally hand-edit it:

[mcp_servers.archiver-rag]
command = "/path/to/archiver-rag"
args = ["serve"]

[mcp_servers.archiver-rag-http]
url = "http://127.0.0.1:8077/mcp"

Verify with codex mcp list. If your server sits behind a proxy that wants a token, --bearer-token-env-var NAME reads it from the environment — archiver-rag itself never checks it (see below).

Reaching it from another machine

archiver-rag performs no authentication and terminates no TLS. Anyone who can reach the port has full access: the entire vault is readable, and log_note and move_notes can modify it.

It is deliberately not this tool's job to decide how you secure that. Keep the server on loopback and put a layer you already trust in front of it — a reverse proxy that terminates TLS and handles authentication, an SSH tunnel, or a VPN (for example Tailscale, kept private to your tailnet). The server does not need to know which; it stays on plain HTTP at 127.0.0.1:8077 in every case.

Anything that publishes the server to the public internet — Tailscale Funnel, a public tunnel, an open port — makes your vault readable and writable by anyone who finds the URL. Only do that behind a layer that authenticates requests.

If you bind beyond loopback (--host 0.0.0.0), the CLI prints a warning — heed it. You can additionally enable DNS-rebinding protection by naming the hostnames you expect to serve:

archiver-rag serve --transport http --allowed-host vault.internal.example

Requests arriving with any other Host header are rejected with 421. The bind address itself is always allowed, so local access keeps working.

Options

http_host, http_port and http_path in config.json supply defaults for the corresponding flags.

Agent instructions (skills)

Registering the MCP server gives an agent access to the tools — but agents tend to fall back on their own internal memory instead of reaching for the vault. The instruction files in skill/ fix that: they enforce a vault-first rule so the agent searches and stores knowledge in your vault before anything else.

What the skill enforces:

• Before answering or reading source files — call search_vault first; only fall back to internal memory if the vault returns nothing relevant
• When something important is missing from the vault — proactively log_note it. If a fact, decision, or piece of context matters to the overall picture and a search_vault came back empty, record it so the knowledge graph grows instead of letting that context die in a single session
• After solving a non-trivial problem — call log_note to capture the decision/lesson/gotcha back into the vault
• The vault is the authoritative memory system — internal agent memory is a fallback only

A version is provided for each agent, since each loads instructions differently:

Each file is self-contained — it includes the MCP registration snippet for that agent plus the full vault-first rules and tool reference. For Claude Code the file is an on-demand skill; for the others it's an always-on instruction file (loaded into every session), which makes the vault-first behavior unconditional.

CLI reference

archiver-rag init                      # one-time setup wizard
archiver-rag start                     # start the background watcher service (default target)
archiver-rag start http                # install + start the detached MCP HTTP server (launchd/systemd)
archiver-rag stop [watcher|http]       # stop the watcher or the HTTP server
archiver-rag restart [watcher|http]    # restart the watcher or the HTTP server
archiver-rag status                    # service liveness, watcher activity, index drift, config
archiver-rag status --json             # same report as JSON
archiver-rag index                     # force re-index the entire vault (runs prune_orphans)
archiver-rag sync                      # ingest only new/modified notes + prune orphaned chunks
archiver-rag prune                     # remove index chunks whose source file no longer exists
archiver-rag search "query"            # test semantic search from the terminal
archiver-rag health                    # index-vs-disk drift + vault health (orphans, broken links)
archiver-rag health --json             # same report as JSON
archiver-rag logs                      # tail the service log

# Knowledge logging
archiver-rag log "Title" --type decision --tag arch --related NoteA
archiver-rag delete <note>... [--yes]  # move to .trash/ + sweep inbound wikilinks (recoverable)

# Folder descriptions (drive semantic placement)
archiver-rag describe                  # generate missing _folder.md descriptions
archiver-rag describe --all            # regenerate all source:auto descriptions
archiver-rag describe --folder <folder> [--set "term1 term2"]   # one folder; --set marks source:manual

# Placement & clustering
archiver-rag place <note>              # suggest folder (semantic + type fallback)
archiver-rag place <note> --apply      # move the note immediately
archiver-rag place --all               # dry-run: current vs suggested folder for every note
archiver-rag place --all --apply       # batch move all notes to their suggestion
archiver-rag cluster [--min-size 2] [--apply]   # [EXPERIMENTAL] label propagation over the wikilink graph (manual/diagnostic only)
archiver-rag relink                    # dry-run: report ## Related before/after under the margin rule
archiver-rag relink --apply            # one-time repair: rebuild every note's ## Related section

# Config
archiver-rag config --auto-cluster / --no-auto-cluster        # watcher auto-placement on/off
archiver-rag config --cluster-threshold 5                     # deprecated — no effect, kept for old configs
archiver-rag config --placement-threshold 0.55                # cosine threshold for semantic placement
archiver-rag config --type-fallback / --no-type-fallback      # fall back to frontmatter type: field
archiver-rag config --auto-describe / --no-auto-describe      # watcher regenerates _folder.md on membership change

archiver-rag uninstall         # remove all data, both services, and MCP registration

start http takes the same transport flags as serve --transport http (--host, --port, --path, --stateful, --allowed-host), bakes them into the installed service, and asks at runtime whether to start automatically at login (--login / --no-login to script it). It refuses to start if the port is already bound. stop http / restart http manage the same service; uninstall removes both services.

MCP tools (for agents)

Once registered, agents have access to 6 tools:

Whole-vault label-propagation clustering (formerly the cluster_vault MCP tool) is manual/diagnostic-only now — see the experimental archiver-rag cluster CLI command above.

Configuration

All runtime config lives at the XDG config path (~/.config/archiver-rag/config.json on Linux/macOS, resolved by archiver_rag/paths.py):

{
  "vault_path": "/path/to/your/vault",
  "install_path": "/Users/you/.local/share/archiver-rag",
  "chroma_path": "/Users/you/.local/share/archiver-rag/chroma_db",
  "auto_cluster": false,
  "placement_similarity_threshold": 0.55,
  "type_fallback": true,
  "auto_describe": false,
  "auto_inbox": false,
  "http_host": "127.0.0.1",
  "http_port": 8077,
  "http_path": "/mcp"
}

{
  "advanced": {
    "term_extraction_min_notes": 4,
    "alpha_curve": {"type": "log", "scale": 1.0},
    "mmr_lambda": 0.5,
    "max_terms": 6,
    "tag_terms_in_description": true,
    "placement_weights": {"identity": 0.6, "content": 0.4},
    "name_prefix_bonus": 0.15,
    "folder_vacancy_grace_periods": 3,
    "link_margin": 0.05,
    "max_total_links": 15,
    "inbox_min_cluster_size": 3,
    "inbox_similarity_threshold": 0.5
  }
}

Data (the ChromaDB index and centroids.json, the per-folder description centroid cache) lives at ~/.local/share/archiver-rag/, and the cache dir at ~/.cache/archiver-rag/ holds runtime.json — the watcher's heartbeat that archiver-rag status reads to report activity, counters, and crash loops. An existing pre-XDG ~/.archiver-rag/ install is migrated automatically and non-destructively the first time any archiver-rag command runs — the old directory is left in place, never deleted.

How configuration works

• Changing settings. archiver-rag config sets the common switches: --auto-cluster/--no-auto-cluster, --placement-threshold, --type-fallback/--no-type-fallback and --auto-describe/--no-auto-describe. Everything else (auto_inbox, the http_* settings, advanced) is edited in config.json directly.
• Missing settings use their defaults. The values in the examples above are the defaults, so your file only needs the settings you want to change.
• When changes take effect. The watcher reads the file again for every note it processes, so placement, description and inbox settings apply to the next note you save, with no restart. The HTTP settings apply after archiver-rag restart http. CLI commands read the file each time they run.
• If the file is invalid (for example broken JSON), everything that moves or creates notes (auto_cluster, auto_describe, auto_inbox) switches off until it's fixed, so a typo can never start reorganizing your vault. Commands that need your vault path stop with an error.
• Checking what's in effect. archiver-rag status (or status --json) shows the placement settings exactly as the watcher applies them, defaults included.

Key by key:

• auto_cluster — off by default; when on, the watcher runs semantic placement (suggest_folder) for new notes and moves the file to the best-matching described folder. It never runs full-graph label-propagation clustering automatically — that stays a manual, experimental archiver-rag cluster command.
• placement_similarity_threshold — cosine similarity a folder must clear to win semantic placement (0–1).
• type_fallback — when no folder clears the threshold, place the note in its frontmatter type: folder instead.
• auto_describe — the watcher regenerates a folder's _folder.md description (blended adaptively, never touching source: manual) whenever a note is created, deleted, or moved in or out of it.
• auto_inbox — route notes no folder claims into inbox/, and spin a new described folder out of it once inbox_min_cluster_size embedding-similar notes accumulate. Off by default.
• http_host / http_port / http_path — defaults for the HTTP transport, shared by serve --transport http and start http.
• advanced — optional tuning knobs, see Advanced tuning above.

Deprecated: cluster_threshold (and archiver-rag config --cluster-threshold) no longer has any effect. It only mattered for the whole-vault clustering fallback the watcher used to run, and is still accepted so older configs load without errors.

The knowledge graph model

The vault is treated as a knowledge graph, not a file hierarchy. Notes are nodes; wikilinks are edges. Relationships range from tight (direct links) to loose (semantic proximity surfaced by search).

Note types are expressed through frontmatter, not folder structure:

---
type: decision
tags: [architecture, async]
related:
  - AsyncLocalStorage
  - PrismaExtensions
date: 2026-04-27
---

The ## Related section at the bottom of each note is managed automatically by the auto-linker after every ingest. Don't edit it manually — it will be overwritten.

Roadmap

Features on the way:

• RAG-Anything integration — extend ingestion beyond Markdown to handle PDFs, Office documents, images, and other file types, so the vault can become a true multi-format knowledge base rather than .md-only.
• Archiver subagents — dedicated subagents that take over vault management (search, logging, reorganization, clustering) on the main agent's behalf, so the primary agent can delegate knowledge work instead of context-switching into it.

License

MIT