docir 0.12.0

documents as IR — a CLI that compiles git-backed markdown
into a verifiable, read-optimized index for AI coding agents.

The idea · Quickstart · Why not just… · Commands · Upgrading · Docs · Live site

The idea

"IR" is intermediate representation — the thing a compiler turns source code into. docir treats your markdown the same way: the files are the source, and the SQLite index (metadata + FTS5 full-text + a typed relation graph + semantic embeddings) is a derived artifact you can throw away and rebuild.

  source of truth     docir                  derived index
  canonical           the compiler           rebuildable · gitignored
  ───────────────     ──────────────────     ─────────────────────────
  decisions/*.md      parse · validate       metadata · FTS5
  issues/*.md     ──▶ allocate ids       ──▶ relation graph (typed)
  tags.yaml           embed (deferred)       vector embeddings

Git is canonical. docir reindex rebuilds the entire index from the files. When the database and the files disagree, the files win.

Why not just…

Orientation, not a shoot-out — the right tool depends on your setup.

† What semantic retrieval costs you

Semantic search runs on a real embedding model, installed by default. It is quantized, CPU-only, and runs locally — nothing is sent anywhere — but it is not free:

If that is too heavy — a CI image, a container you keep small, an air-gapped box — opt out and docir falls back to a dependency-free hashing embedder:

export DOCIR_EMBEDDER=deterministic

That embedder scores similarity by shared vocabulary rather than meaning, which is the same signal the full-text index already provides. The cost is measured, not asserted: docir context scores recall@5 0.96 with the model against 0.93 without it, and puts the right document first far more often (MRR 0.95 vs 0.80). Isolate the embedding signal by turning graph expansion off, and on questions phrased in words the documents never use the model gets 0.86 where the fallback gets 0.79 — below the 0.79 that plain full-text search manages on its own. Corpus, tasks, judgments and caveats are in benchmarks/; uv run python benchmarks/run.py reproduces it.

Long documents are embedded per section

The model reads about 512 tokens — roughly 1,900 characters — and silently ignores the rest. Not downweights: ignores. Append a sentence past that point and the vector comes back bit-identical. 84 of the 103 documents in docir's own store are longer than that, so 56% of the corpus was not in the semantic index at all — and nothing said so, because full-text search covers the whole body and rescued the rank on any query that shared a word with the document.

docir therefore embeds each ## section as well as the whole document (adr-927aa43d9635), and a document ranks on its best-matching section. Coverage on docir's own store: 44% → 100%. On the same corpus, context recall@5 holds at 0.97 while MRR rises 0.94 → 0.97. benchmarks/run.py reports the coverage figure and measures the window empirically, so it stays honest if the model changes.

Reading follows ranking: if context surfaced a document for one of its sections, the hit says which — matched_section carries that heading, ready to read back.

docir get arch-1cfb1b212237 --section "Daemon process"

It returns the same span update --replace-section would overwrite, and an unknown heading errors listing the ones that exist.

Switching embedders re-embeds rather than mixing vector spaces: docir records which model produced each vector, ignores the others, and recomputes them on the next write or docir embed --flush.

Quickstart

# 1. install
uv tool install docir          # or: pipx install docir

# 2. scope docs to this repo (creates ./.docir, like `git init`)
#    skip it and docs go to the global ~/.docir — docir warns if you are in a repo
docir init

# 3. teach this repo's AI agent to drive docir (writes a Claude Code skill)
docir agent install            # add --agent agents for an AGENTS.md block
                               # (or, for an MCP client: docir mcp serve)

# 4. capture a decision…
docir add --type decision --title "Auth strategy" \
    --description "How the service authenticates API clients." --stdin < draft.md

# 5. …and retrieve it by intent, next session
docir context "implement a new auth endpoint"

In a terminal, docir context prints ranked, body-less skeletons — frontmatter and typed edges, no body — so you scan wide, then fetch a body by id with docir get:

$ docir context "implement a new auth endpoint"
┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃ id         ┃ type     ┃ status   ┃ title              ┃ description                      ┃ score ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│ adr-0001   │ decision │ proposed │ Auth strategy      │ How the service authenticates    │ 0.033 │
│            │          │          │                    │ API clients.                     │       │
│ issue-0001 │ issue    │ open     │ Token refresh race │ Refresh token race under         │ 0.016 │
│            │          │          │                    │ concurrent logins.               │       │
└────────────┴──────────┴──────────┴────────────────────┴──────────────────────────────────┴───────┘

Built for agents, though: when the output is captured (stdout isn't a TTY), the same command emits compact, trimmed JSON — no borders, empty fields dropped, ~40% fewer tokens:

$ docir context "implement a new auth endpoint" | cat
[{"id":"adr-0001","title":"Auth strategy","description":"How the service authenticates API clients.","type":"decision","status":"proposed","tags":["auth"],"archived":false,"stale":false,"score":0.0328,"similarity":0.8951,"via_graph":false}, ...]

An absent field means its default (no owner, not stale). score is a reciprocal-rank fusion of the full-text and vector rankings, so ordering is the point and the absolute value means little — a nonsense query against a one-document store scores about the same as a perfect match. similarity is the raw cosine against your query and does carry absolute meaning, which is what --min-score filters on: with it, an empty result is a real answer rather than an impossible one. --json forces JSON anywhere, --pretty forces the table, --no-trim keeps every field.

Scope and limits

• Search covers title, description and body — not tags. Tags are a controlled vocabulary for docir query --tag, deliberately kept out of the full-text index so one tag match cannot flood out the text matches. docir search auth will not find a document merely tagged auth.
• List paths page. query, search and tag list take --limit and --offset, applied in the query rather than after it. A page shorter than --limit means the end; there is no total, because the response is a bare JSON array.
• context is not paged, by design. It returns a minimal relevance-ranked set bounded by --limit — a token budget, not a browse path. It does load every current embedding per call, which is what sets the practical corpus ceiling.
• Dates are UTC calendar dates. created, updated and verified are written into committed files and read by other people, so they do not depend on the writer's timezone.

The model

• Git is the source of truth. The index is a compile artifact — derived, .gitignored, rebuildable. Nothing lives only in the database.
• One write path. Agents never edit markdown directly; every write goes through the CLI, which guarantees frontmatter/schema consistency and collision-free id allocation. You are not an agent: the files are yours, and the rule for humans is narrower — see what you may edit by hand.
• Reads return skeletons. query / search / context return frontmatter + typed edges + staleness — no body. Fetch bodies by id with get, or a single section with get --section. An agent scans wide cheaply, then reads deep only where it matters.
• Staleness is data, not a guess. Optional owner / verified fields plus a per-type review cadence make "is this doc still true?" a first-class, checkable fact — and a worklist: docir query --owner platform-team --stale is one steward's review queue, cleared a document at a time with docir update <id> --verified.
• Relations are typed. A related edge carries a kind (supersedes, depends_on, implements, …) — a real graph, not a bag of links.
• A document can name the code it governs. Optional code globs (docir add --code "src/auth/**") record which files a decision is about. Only the shape is validated on write — a decision may be written before the code it decides — and docir check warns when a pattern stops matching anything, so a decision whose code moved is visible rather than quietly wrong. Ask it in reverse with docir query --code src/auth/login.py: which decisions govern the file I am editing.
• A decision that can be enforced is enforced by a test. Point code at the test that fails when the code contradicts the decision, and CI already enforces it — in your language, with your fixtures. docir records the link and warns when that test disappears; it ships no rule engine (adr-b2cfed9d5888). At review time, docir query --code $(git diff --name-only origin/main...HEAD) lists what a branch should be read against — a notice, not a gate.

What you may edit by hand

The files are git-backed markdown and docir reindex exists precisely to pick up an outside change — so hand-editing is supported, but not on every field:

Then run docir reindex && docir check — or let the daemon do the reindex for you. It watches .docir/docs/ and rebuilds what changed within a second of the edit, which is safe precisely because the files are canonical: a reindex only makes the index agree with them, and writes no markdown. DOCIR_WATCH=0 turns it off; --no-daemon runs never watch, so CI still needs the explicit command. docir check is still yours to run. Reindex reports documents_skipped for files whose frontmatter will not parse — those are absent from every read path, not merely flagged — and check catches unregistered tags, undeclared statuses, unknown types, dangling links and duplicate ids. A hand-written verified date is the one thing nothing can verify, which is exactly why it should not be written by hand.

Commands

Full command reference

init · add · update · archive · unarchive · delete
get · query · search · context · build
tag {add, list, rename, rm}
agent {install, update}
schema {show, validate}
self {status, upgrade}
check [--fix] · lint · reindex · embed · version
daemon serve · mcp serve

Store precedence (highest first): --home → DOCIR_HOME → a project-local .docir/ found by walking up from the CWD → the global ~/.docir default. --no-daemon runs any command in-process instead of over the daemon socket. Output is a Rich table at a TTY and compact JSON when piped; --json / --pretty force either, and --no-trim keeps every field. That applies to --help too — docir --help | cat returns the command vocabulary as JSON, so an agent can discover the CLI without parsing box-drawing characters.

Two ways an agent reaches docir

Some agents run shell commands; some only call MCP tools. Both get the same vocabulary, because both go through the same dispatcher — an MCP tool and its CLI command cannot answer differently.

docir agent install               # a Claude skill / an AGENTS.md block: drive the CLI
claude mcp add docir -- docir mcp serve   # or the same commands as MCP tools

docir mcp serve speaks stdio (what an MCP client spawns) or --transport http, and needs no extra — the MCP server ships with docir, because an agent that only speaks MCP cannot install the extra it would need to reach docir in the first place. uvx docir mcp serve runs it without installing anything.

The tools are named docir_context, docir_get, docir_add, … Reads carry the readOnlyHint annotation and return the same body-less skeletons the CLI does, trimmed the same way; requests go through the daemon by default, so the embedding model stays warm across calls.

For the humans who approve the decisions

An agent reads docir through the CLI or MCP. The people who have to approve a decision usually are not at a terminal, and a decision only an agent can read is a hard sell to them.

docir build --out site/                             # one page per document, plus an index
docir build --out site/ --title "Acme — decisions"  # names the heading, the tab and the wordmark
docir build --out site/ --logo brand/mark.svg       # your logo in the corner and the tab
docir build --out site/ --include-archived          # publish archived documents too

The result is self-contained — inline CSS, no external requests — so it opens from file:// and publishes to GitHub Pages or S3 unchanged. docir publishes its own store that way — browse it live at l0kifs.github.io/docir — from .github/workflows/pages.yml; copy it, and enable Pages once under Settings → Pages → Source: GitHub Actions. Reindex first — .docir/docs/ is committed and the index is gitignored, so a fresh clone has none and build would otherwise publish an empty list (it warns, and the workflow gates on the page count). It shows what only docir knows: the typed relation graph in both directions (a superseded decision says so, in a banner, linking the one that replaced it), the staleness flag, tags, owner and dates. --title is what the site calls itself — the heading, the browser tab and the name beside the mark; without it every page reads "Documentation". --out is regenerated on every build, so a document deleted from the store cannot survive as an orphaned page; a directory docir did not build is refused unless you pass --force.

How state is stored

State lives in one resolved store per invocation. Run docir init in a repo to keep its docs with the code: .docir/docs/ and docs-schema.yaml are committed; the derived index (SQLite + embeddings) is gitignored and rebuilds with docir reindex. Without init, docir falls back to a global ~/.docir.

The daemon keeps the embedding model warm and serializes writes; the CLI is a thin, stateless client that spawns and respawns it transparently. Embeddings are the one deferred, eventually-consistent piece — a content change flags the vector dirty and returns; everything else (file, metadata, FTS, relations) is synchronous. Force a flush with --wait-embeddings, docir embed --flush, or docir reindex --embeddings.

Upgrading

The schema, the agent instructions and the site templates ship inside the package, so a release can change what a store enforces and what an agent reads with nothing in your git diff to review. Migrations, a daemon serving old code (it records the build it loaded and is respawned once that stops matching) and vectors from a superseded model all sort themselves out on the next command. What is left is one command:

docir self upgrade        # install the new docir, then resync this store
docir self status         # what is installed, and whether anything newer exists

self upgrade installs the newest docir where docir owns its environment — a uv tool, a pipx install, a virtualenv — then re-executes as the build it just installed and does the rest: rebuild the index (derived and gitignored, and the only place the schema baseline and the version that built it are recorded), refresh any installed agent instruction file, and report what check still finds, in that order. Where docir does not own its environment — a checkout, a project whose lockfile pins it, an ephemeral uvx run — it says so and resyncs the store anyway: that package belongs to the project, not to you. --no-package skips the install.

The re-exec is the point of the ordering: the process that runs the installer is the code being replaced, so everything after it has to be the new build's work — starting with the stamp that records which version built the index.

Until a store is rebuilt it reports neither schema-drift nor stale-index-build: absent means unknown, not unchanged. That is also what a fresh clone needs — the index is gitignored, so a clone has none, and an empty index answers no structural issues exactly like a healthy one.

docir makes exactly one network call in its life, and only if you ask: self status --refresh looks up the newest release, at most once a day. DOCIR_UPDATE_CHECK=1 has the daemon keep that answer fresh and every command mention a newer release on stderr; it is off by default, because a notice that repeats until you act on it stops being read.

The rest of the procedure — docir init --force, MCP clients, pinning the version CI installs — is a runbook in docir's own store.

Schema: core + profiles

Documents are constrained by a per-type schema (required fields, status grammar, allowed relations). docir ships a frozen, domain-agnostic core plus swappable profiles — software (default: decision / issue / architecture / release_note), research, ops, qa, legal. A docs-schema.yaml merges core → profiles → inline, so you extend it without mutating the base.

docir init --profiles software,qa   # pick profiles up front
docir init --id-style sequential    # readable adr-0007 instead of the default random
docir schema show                   # the merged result — what validation enforces
docir schema validate               # check an edit before it reaches a write

docir init writes id_style: random by default — ids like adr-3f9a2b1c7d4e, which two branches can never mint identically. Pass --id-style sequential for human-friendly adr-0007 numbering; it is collision-free within one store, but a merge can bring two branches that each allocated the same number (docir check reports it as duplicate-id).

The generated docs-schema.yaml carries a commented-out worked example of the inline types: / relation_types: syntax, so the grammar is discoverable at the point of use.

Architecture

Vertical bounded-context modules (documents, tags, indexing, agents) over a shared platform, wired by thin entry_points. Dependencies flow entry_points → modules → platform → config; boundaries are enforced by tach in CI — not by convention. Each module exposes exactly one public file (api.py) described by a CONTRACT.md.

The design rationale and the module rules are themselves docir documents — run docir get arch-1cfb1b212237 and docir get arch-322e5f992ad2, or browse .docir/docs/architectures/. docs/README.md maps every pre-migration path to its id.

Contributing

Issues and PRs welcome. docir dogfoods itself: its ADRs, architecture documents, runbooks and gap register live in its own store, so docir context "what you are about to change" is how you orient. Read the architecture rules and the ADRs (docir query --type decision) first — module boundaries are machine-checked by tach in CI, alongside lint, type-check, and a coverage gate. Every design deviation is recorded as an ADR, added with docir add --type decision, never by hand.

uv sync                                              # dev environment
uv run python benchmarks/run.py                      # retrieval quality + token cost
uv run pytest --cov=docir --cov-fail-under=90        # tests + coverage gate
uv run ruff check . && uv run ty check && uv run tach check

License

MIT © Sergei Konovalov