refs-mcp 0.2.0

FastMCP server for managing the refs/ reference-repo tree

refs

You have a ~/dev/refs/ folder too, don't you?

The one where every time you want to read someone's codebase — study drizzle-orm's query builder, skim next.js internals, grep through playwright — you run

git clone https://github.com/foo/bar.git ~/dev/refs/bar

and then months later you've got a hundred loose checkouts at the top level, half of them stale, two of them accidentally the same repo under different names, and no idea which ones are your work vs. someone else's code. You keep meaning to organize it. You never do. Nobody wants to write a housekeeping tool for their own scratch directory.

This is that tool.

What it does

Run ./reorg.sh --apply and your pile becomes this:

refs/
├── facebook/
│   └── react/
├── vuejs/
│   └── vue/
├── microsoft/
│   ├── playwright/
│   └── ...
└── ... (one directory per GitHub owner)

No matter where a clone was before — flat at top, nested in a mis-named folder, duplicated under two names — it ends up at <owner>/<repo>/ based on its actual git remote. Duplicate clones of the same upstream get parked aside for review instead of clobbering. Random non-repo content (your notes, binaries, scratch dirs) gets evicted out of the tree entirely. Empty pseudo-owner folders get pruned.

Every move is journaled. ./reorg.sh undo --apply reverses the most recent run. Nothing magical — plain mv, rmdir, git.

The daily loop

./reorg.sh --apply     # you cloned some more stuff; put it where it belongs
./status.sh            # quick look at what's dirty and what has updates
./update.sh            # git pull every clean repo with remote changes

About the git repo you're looking at

This directory is itself a git repo, but it doesn't track the hundreds of cloned reference repos — those are checkouts you can always re-clone. It only tracks the tooling (reorg.sh, status.sh, update.sh) plus the two generated indexes (index.md, index.html). A whitelist-mode .gitignore enforces this. More on that below.

Tools

reorg.sh

Reorganizes loose clones into the <owner>/<repo>/ layout, keeps an index (index.md, index.html) up to date, evicts non-repo content out of the collection, and journals every move so you can reverse it.

./reorg.sh                    # dry-run preview
./reorg.sh --apply            # perform moves, write journal
./reorg.sh undo --apply       # reverse the most recent apply
./reorg.sh --help             # full option list

Destinations (overridable):

purpose	default path
duplicate-URL losers	$(dirname $ROOT)/!CONFLICTS-RESOLVE/
non-repo top-level content	$(dirname $ROOT)/!WORK-PRODUCT-FROM-REFS/
apply journals	${XDG_STATE_HOME:-~/.local/state}/reorg/<basename-of-ROOT>/

status.sh

Reports dirty/clean + has-updates for every repo in the tree. Walks depth 1 and 2, so both loose-at-top and <owner>/<repo>/ layouts are covered.

update.sh

git pull every clean repo that has remote changes. Skips dirty repos with a note.

test-reorg.sh

Integration test harness for reorg.sh. Runs shellcheck, then 42 tests over synthetic git fixtures (fake .git/config files — no network, no real repos needed).

./test-reorg.sh                 # full suite
./test-reorg.sh --no-shellcheck # skip the lint step

MCP server (FastMCP)

refs_mcp/ ships a FastMCP server that exposes the same operations as MCP tools, with structured outputs typed by Pydantic. Useful when an agent wants to inspect or operate on the tree through the Model Context Protocol instead of shelling out.

Install + run

uv sync --frozen                      # install deps from uv.lock
uv run --frozen refs stdio        # serve over MCP stdio
uv run --frozen refs http         # serve over streamable-HTTP (daemon)
uv run --frozen refs --help       # full CLI surface

The CLI is a click group with two transport subcommands — stdio for Claude Desktop / IDE clients that spawn the binary, and http for daemons under systemd / Docker / k8s. Anchored to the canonical pattern in ../refs/modelcontextprotocol/servers/src/git/ and ../refs/github/github-mcp-server/.

Or build a frozen binary and invoke it directly:

uv run --frozen python -m refs_mcp._build     # produces dist/refs.exe
./dist/refs stdio                              # no Python needed at run time

The build wrapper installs a logging.Handler on PyInstaller's logger that sys.exits on every record at WARNING or above — warnings genuinely fail the build. There is no --werror flag in PyInstaller or uv (verified via JiT reads of both upstreams).

The canonical dev commands use uv run --frozen so dependency resolution is locked to uv.lock and uv never rewrites the lockfile as a side effect:

uv run --frozen pytest tests/                  # 177 tests
uv run --frozen ruff format refs_mcp tests     # format
uv run --frozen ruff check  refs_mcp tests     # lint
uv run --frozen pyright refs_mcp               # static type check
uv run --frozen refs selftest                  # live MCP smoke (in-process)
uv run --frozen refs selftest --binary dist/refs.exe   # vs the frozen binary

CI is a three-tier pipeline in .github/workflows/ci.yml, fronted by a tiny changes gate. Capability is preserved across tiers — macOS, Python 3.11/3.13, PyInstaller build, MCP stdio smoke, artifact upload, and release upload are all defined in the workflow. What varies per event is which tier fires AND whether the cheap default tier fires at all on PR/branch-push (a docs-only change skips it entirely — Windows never starts).

Event	changes	test	test-full	build	release
pull_request to main (code change)	✓	✓	—	—	—
pull_request to main (docs only)	✓	—	—	—	—
push to main (code change)	✓	✓	—	—	—
push to main (docs only)	✓	—	—	—	—
workflow_dispatch with full-ci: true	—	✓	✓	✓	—
push tag v*	—	✓	✓	✓	✓

changes is a cheap ubuntu probe (dorny/paths-filter@v4) that flags whether this push touched any runtime-relevant file: refs_mcp/**, tests/**, pyproject.toml, uv.lock, refs.spec, .github/workflows/ci.yml, or any *.sh. If only docs/comments/README changed, every downstream tier skips and the PR's required check passes without spinning up Windows. The gate itself is bypassed on workflow_dispatch + tag push (the operator explicitly asked for the full run).

test is the default cheap signal — ubuntu-latest and windows-latest on Python 3.12 only. Linux runs ruff + pyright + shellcheck + bash test-reorg.sh + pytest; Windows runs ruff + pytest. No macOS, no extra Python versions.

test-full is extra source-pytest coverage: {ubuntu, windows} × {3.11, 3.13} + macos-latest/3.12. Source pytest only, no extra gates (pyright + shellcheck are version-independent and already covered by the canonical test cell).

build produces per-OS PyInstaller binaries pinned to py3.13 (the shipped interpreter), runs a CLI smoke + the full 6-tool live MCP stdio smoke against three shallow-cloned target repos (MicrosoftDocs/mcp + BurntSushi/ripgrep + modelcontextprotocol/python-sdk), and uploads each binary as an artifact (retention 7 days for non-tag runs).

release downloads the three per-OS binaries and attaches them to a GitHub release via softprops/action-gh-release@v3.

To run the full sweep manually (e.g. before cutting a release), use the Actions tab and dispatch the ci workflow with full-ci: true.

Set REFS_ROOT to point at a tree other than the current directory. REFS_CONFLICTS_DIR, REFS_WORK_PRODUCT_DIR, and REFS_JOURNAL_DIR override the matching reorg.sh defaults.

Add to a Claude Code project's .mcp.json:

{
  "mcpServers": {
    "refs": {
      "command": "uv",
      "args": ["run", "refs", "stdio"],
      "cwd": "<absolute path to your refs/ tree>"
    }
  }
}

Or, when the frozen binary is on PATH:

{
  "mcpServers": {
    "refs": {
      "command": "refs",
      "args": ["stdio"],
      "cwd": "<absolute path to your refs/ tree>"
    }
  }
}

Tools

Evidence-graded search + symbol extraction (read-only by construction):

Tool	Read-only	Implementation
refs_find_repo	yes	Bounded-scope repo lookup by owner/repo, exact name, or fuzzy substring.
refs_search_evidence	yes	rg --json content search with verdict labels (MATCH / VALIDATED_EMPTY / INVALID_EMPTY / TRUNCATED / FAILED / SKIPPED_UNSAFE).
refs_prove_absence	yes	Single-pass literal probe + positive control; strict VALIDATED_EMPTY or FAILED.
refs_inspect_terms	yes	Batch verdict map over a list of terms in one repo.
refs_list_symbols	yes	Language-agnostic symbol extraction via tree-sitter-language-pack (306 languages) with DEFINED_AND_TESTED / DEFINED_ONLY / DEFINED_PRIVATE verdicts.
refs_find_symbol	yes	Resolve one symbol's signature + tests + head SHA across any supported language.
refs_host_tools	yes	Probe-based inventory of host CLIs (rg, git, gh, …) + Python-module gates (tree-sitter) with feature-gate map.
refs_discover_remote	yes	gh search repos for cross-org discovery (needs gh on PATH).
refs_discover_remote_org	yes	gh repo list <owner> for enumerating an org's repos.
refs_preseed_run	no	Auto-clone the curated upstream-reference set (FastMCP, MCP SDK + spec, Pydantic, ripgrep, OTel semconv, pgr, MicrosoftDocs/mcp).

Symbol extraction (language-agnostic)

refs_list_symbols and refs_find_symbol route per-file to an extractor stack modeled on the Sourcegraph + GitHub code-nav architecture:

Tier	Extractor	Status
primary	tree-sitter via tree-sitter-language-pack (306 languages)	shipping
enrichment	SCIP / LSP semantic indexes — override structural records with semantic-precise resolution (overloaded / imported / external symbols).	protocol locked; future work
fallback	Universal Ctags subprocess — for languages tree-sitter doesn't ship a grammar for	protocol locked; future work

The Symbol IR carries name, kind (function / async_function / method / class / struct / enum / trait / interface / type_alias / impl / constant / variable), language, signature, line, end_line, visibility, parent, decorators, extractor (which extractor produced this record — useful for verdict explainability), plus the DEFINED_AND_TESTED / DEFINED_ONLY / DEFINED_PRIVATE / NOT_FOUND verdict.

Across the smoke targets, tree-sitter extracts:

Repo	Language	Symbols
modelcontextprotocol/python-sdk	Python	631
MicrosoftDocs/mcp	TypeScript	100
BurntSushi/ripgrep	Rust	3472

Tree management (every tool runs in native Python or git/gh subprocess — no shell wrappers):

Tool	Read-only	Implementation
refs_help	yes	Returns the typed HelpDocument — read this first per session.
refs_status	yes	Filesystem walk + configparser on .git/config. Offline.
refs_discover	yes	Same walk; optionally writes $REFS_JOURNAL_DIR/refs.inventory.json.
refs_reorg_preview	yes	Pure planner — builds ReorgPlan from typed inventory, no IO.
refs_reorg_apply	no	pathlib + shutil.move + csv.writer for the TSV journal.
refs_reorg_undo	no	csv.reader reads the journal; moves reversed with shutil.
refs_clone	no	gh repo clone (github.com, auth-aware) or git clone; dry-run default; refuses non-GitHub and overwrites.
refs_update_check	yes	git status --porcelain=v2 --branch + git ls-remote for has_updates.
refs_update_apply	no	git stash push → git pull --ff-only → git stash pop; per-repo concurrency.
refs_degit_export	no	git archive into an export dir; source clone untouched.
refs_sparse_materialize	no	git sparse-checkout set; traversal patterns refused.
refs_index_generate	no	Pure-Python renderer of index.md + index.html from typed Inventory.
refs_journal_latest	yes	csv.reader over the latest TSV journal.
refs_events_tail	yes	Read the JSONL operation events log.

Feature gates (host_tools probe)

Each tool that depends on a host CLI is gated by the bootstrap host_tools probe — shutil.which is not enough; the probe runs each tool's --version and only marks it OK on success. Features that need an absent tool return a typed FAILED verdict with the install hint, not a silent zero. See refs_host_tools for the live snapshot.

Per-feature gates land in refs_mcp.host_tools._CURATED_FEATURES:

Feature	Required tool(s)	Used by
content_search	rg	refs_search_evidence, refs_prove_absence, refs_inspect_terms
file_enum	rg	Positive-control probes, corpus building
structural_symbols	Python tree_sitter_language_pack	refs_list_symbols, refs_find_symbol (gated on Python import, not a CLI)
auto_clone	gh (preferred) or git	refs_mcp.auto_clone.ensure_repo
git_status	git	refs_update_check sweep
git_pull / git_stash	git	refs_update_apply sweep
git_archive_local	git	refs_degit_export
git_sparse_checkout	git	refs_sparse_materialize
remote_repo_search	gh	refs_discover_remote
remote_org_list	gh	refs_discover_remote_org
remote_archive_tarball	gh	Future: archive without local clone (gh api repos/{o}/{r}/tarball/{ref})
remote_metadata	gh	Default-branch / head SHA / disk size without clone
pre_clone_size_inspection	gh	refs_discover_remote populates DiscoverableRepo.size_kb

~/.refs/ layout (XDG-friendly)

Bootstrap writes per-run artifacts to a user-level layout (overridable via env vars; defaults follow the XDG Base Directory spec on Linux). Actual on-disk layout as implemented in refs_mcp.user_config.UserPaths:

~/.refs/
├── .refs.toml                  # user preferences (auto_clone_allowed_hosts, etc.)
├── repos/                      # default refs_root (cloned upstreams)
│   └── <owner>/<repo>/
├── state/
│   ├── events.jsonl            # structured operation events
│   ├── server-runs.jsonl       # one entry per server boot (run_id, transport, env)
│   ├── logs/                   # date-rotated structured file logs (trace_id-correlated)
│   └── traces/                 # OTel spans as JSONL per run (file exporter)
├── data/
│   └── replay/                 # captured request/response artifacts per run
└── cache/
    └── symbol-index/           # symbol-index cache keyed by (repo_path, head_sha)

Override individual subtrees with XDG_STATE_HOME / XDG_DATA_HOME / XDG_CACHE_HOME (each gets /refs appended), or the more-specific REFS_STATE_DIR / REFS_DATA_DIR / REFS_CACHE_DIR. Setting REFS_HOME overrides the whole tree — XDG vars are ignored when REFS_HOME is set so the layout stays cohesive (no surprise splits). The config file (.refs.toml) lives directly under the home root; relocate it via REFS_HOME.

The shell scripts (reorg.sh, status.sh, update.sh) remain in the repo for shell-CLI users. The MCP server's reorg/status/update paths do not call them — they share the on-disk contract (whitelist .gitignore, TSV journal format, <owner>/<repo> layout) so a journal written by either path is reversible by the other.

Resources

URI	MIME	Content
refs://inventory	application/json	Structured Inventory of the tree (typed ResourceResult).
refs://inventory/schema	application/schema+json	JSON Schema of the Inventory model.
refs://repo/{owner}/{repo}	application/json	One typed RepoRecord by owner+repo.
refs://config	application/json	Sanitized server config (no secrets).
refs://journal/latest	application/json	Latest reorg journal, parsed.
refs://events/latest	application/x-ndjson	Tail of the structured operation events log (NDJSON).
refs://index.md	text/markdown	index.md rendered from typed Inventory.
refs://index.html	text/html	index.html rendered from typed Inventory.
refs://help	application/json	Typed HelpDocument — same content as the refs_help tool.
refs://help/markdown	text/markdown	Help rendered as markdown for humans.

Prompts

Name	Purpose
refs-agent-onboarding	Read-this-first onboarding for any agent connecting to the server.
audit-refs-tree	Audit-before-mutate workflow.
clone-reference-repo	Safe clone workflow (refs_clone preview → apply → verify).
refresh-reference-tree	status → check → update sequence.
prepare-agent-reference-pack	Surface local upstream docs from the inventory for a given topic.
export-plain-source-snapshot	degit-style snapshot via refs_degit_export.

Help surface (lessons baked into the server)

refs_help and the two refs://help[/markdown] resources expose a typed guide for anyone — operator or agent — about to use this server. The content is generic and transferable: which tools are offline vs network-bound, why generated indexes are never parsed as data, what the journal contract is for undo, why the OpenTelemetry tracer is a no-op without an exporter, why owner/repo names with ., .., control chars, or Windows reserved names are rejected, and which upstream docs are already cloned under the refs root. Tip severities (info / warning / critical) match how aggressively a client should surface them.

Generated indexes are not canonical

index.md and index.html are presentation artifacts produced by reorg.sh. The MCP server's source of truth is the structured inventory built from a filesystem walk + .git/config reads. No tool in this package parses index.md or index.html as data.

Mid-2026 MCP / Claude features in scope

What the server uses today:

• Structured tool outputs and output schemas — every tool returns a Pydantic model, so FastMCP emits both content (legacy text JSON) and structuredContent with a generated outputSchema (MCP 2025-11-25 § Tool Result).
• Tool annotations — read_only_hint, destructive_hint, idempotent_hint, open_world_hint are set on every tool so the client can render safe-vs-destructive UI.
• Resources with MIME — JSON resources are addressable and discoverable via resources/list.
• Prompts — audit-refs-tree teaches an agent the right read-before-mutate workflow.
• Progress + log notifications — refs_reorg_apply and refs_update_apply report bracketed progress to the MCP client via ctx.report_progress and emit structured status via ctx.log. The asyncio streaming runner in refs_mcp.runner is the canonical subprocess entrypoint and remains available for any future tool that needs line-by-line subprocess forwarding.
• OpenTelemetry — FastMCP emits per-call SERVER spans automatically; this package adds a CLIENT-kind span around every subprocess call (git, gh, ripgrep) carrying process.command, process.command_args, process.working_directory, process.exit.code, duration, and ERROR status on non-zero exits. Bootstrap (refs_mcp.bootstrap) wires the TracerProvider once per process with a file SpanExporter writing to ~/.refs/traces/<run_id>.jsonl so every run has captured spans on disk by default. OTLP export is also wired:
  • OTEL_EXPORTER_OTLP_ENDPOINT=… adds the OTLPSpanExporter (proto-http) via BatchSpanProcessor. The exporter reads the standard OTEL_EXPORTER_OTLP_* configuration variables. The dependency ships as a hard runtime dep — no [otlp] extra dance.
  • OTEL_SERVICE_NAME=… (default refs) attaches a meaningful service.name Resource attribute. W3C trace context, semconv service.* / host.* / process.* / os.* attributes are populated by refs_mcp.run_metadata.

What is deliberately not used:

• Sampling — server-requests-LLM. This server doesn't generate text; no natural use case.
• Elicitation — destructive ops are gated by destructive_hint and by the default-dry-run contract; a separate "ask the human" round-trip adds friction without safety.
• Roots — server is launched in a specific working directory (or via REFS_ROOT); root negotiation isn't useful when the path is part of the launch contract.
• Background tasks (execution.taskSupport, SEP-1686) — would require a runner like Docket; advertised once an MVP integrates one.
• Resource subscriptions / subscriptions/listen — useful for watching inventory changes but out of scope for the initial drop.

Tests

uv run pytest tests/

The pytest suite covers path safety, GitHub URL parsing, subprocess runner (including timeout and missing-executable paths), discovery on a synthetic tree, native reorg / git_ops / operations behavior over the git/gh subprocess paths (status-sweep, update-sweep, clone, archive, sparse-checkout), verdict-graded search + symbol extraction, host_tools probe + feature-gate derivation, auto-clone allowlist + path safety, MCP server smoke (tools/resources/prompts list, structured inventory return, annotation presence), OpenTelemetry span emission, and the benchmark harness.

The pre-existing ./test-reorg.sh Bash suite remains authoritative for filesystem moves, conflicts, eviction, and journal/undo semantics.

.gitignore is a whitelist

refs/.gitignore uses whitelist-mode:

/*
!/reorg.sh
!/test-reorg.sh
...

Everything at the top level is ignored except the explicitly un-ignored entries. GitHub owner directories (over a hundred of them, in practice) are correctly excluded — git never sees them, so no embedded-repo warnings, no accidental staging.

To keep a new top-level file, add a !/filename line. reorg.sh's top-level allowlist also reads this file: anything not on the static or dynamic (GitHub owner) allowlist will be evicted to !WORK-PRODUCT-FROM-REFS/ on the next --apply.