honestreview

Review a repo at the granularity its defects actually live at — and never overstate what you checked.

Why

A review can only find defects that fit inside the unit it looks at.

Measured on one codebase: three waves of four-vendor per-file review, ~500 findings, and it missed 13 functions copy-pasted between two files, four writers of one settings file (three destructive — one of them later destroyed the file), and a producer/consumer pair reading and writing different tables.

None of that was carelessness. The reviewer was shown one file at a time, so a two-file defect was never in the room, and no amount of care recovers evidence that is not in the context window. When something is missed, the first question is "was it even in the room?" — not "why wasn't it noticed?"

So "I reviewed the repo" means nothing without the axis:

invariant is skipped most and matters most: it is the only axis that can find something MISSING. "There is no backup before any mutation in this repo" is true, catastrophic, and appears in zero files.

Plus two that cut across all of them: silent (swallowed failures) and unwired (capabilities built and never connected — worse than a bug, because the code looks protected).

A finding is not a finding until something tried to kill it

Raw multi-vendor review over-calls badly. Measured: 60 "dangerous" silent failures became 26 once independent refuters were told to knock each one down and default to refuted when unsure — and 2 after a truncation bug in the refuter itself was fixed.

Three states, kept separate:

• SURVIVED — attempts to refute it failed. A finding.
• REFUTED — the claim did not hold. Correctly closed.
• UNVERIFIED — the check did not complete. Neither of the other two. Collapsing it is how "2 survived, 7 unverified" got reported as "9 survived".

Use

pip install honestreview               # pulls symgrep-codesearch + llm-spendguard + fastmcp from PyPI
pip install "honestreview[learn]"      # + the local sentence-transformers embedder (optional; pulls torch)

# honestreview runs UNDER the spendguard gate, so no axis can ever spend ungated. Arm it once:
spendguard install-hook --venv <your-venv>     # then `spendguard doctor` must print: ENFORCING HERE: YES

honestreview all <root>                # every axis, cheapest-first — ESTIMATE ONLY
honestreview all <root> --run          # actually review
honestreview invariant <root> --run    # the axis that finds what is missing
honestreview ledger                    # every pass, every verdict, what is still open

Search-by-meaning (the concept/seam retrieval) runs in symgrep's own process over MCP/HTTP, so honestreview needs neither an embedding model nor veccore/fastembed in its own environment — point it at a running symgrep-mcp (or set $SYMGREP_MCP_URL). The [learn] extra is only for the optional local doctrine-learning embedder.

Nothing is spent without --run. Every axis prints a cost and a count first. That is not politeness: a review that surprises someone with a bill gets run once and then avoided, and a review nobody runs finds nothing.

As an MCP tool

claude mcp add honestreview -- /path/to/venv/bin/honestreview-mcp

Restart the MCP server after changing doctrine code. Both transports check the package source before each review and refuse stale requests with exit 1 until restarted, including changes that preserve mtimes.

This matters more than the CLI. A toolkit you have to remember is used when it occurs to someone, and what occurs to someone under focus is what they always do — which is how the per-file panel got run three times while the cross-file axes, where every missed defect actually was, were never reached.

As a hook

honestreview install-hook              # backs up settings.json first, atomically

Ships a family of PreToolUse hooks — authorship hooks that read source at the moment it is written (led by the agentic-decision hook, which blocks a mechanical proxy standing in for a judgement) and behavioral hooks that judge a Bash action at the moment it runs. A code hook's matcher includes Bash on purpose — a Write|Edit-only hook is blind to heredoc-written source, which is how most agent-authored files are created. Measured: a rule against regex-decisions sat in three CLAUDE.md files and was violated repeatedly, because every offending file was written through a heredoc. Widening the matcher fixed in one edit what instruction had not fixed in fifty.

See src/honestreview/hooks/agentic_decisions.md for what it catches, the rule that fixed its false positives, and its two known over-fire shapes.

As doctrine

honestreview doctrine >> CLAUDE.md     # the method, without the tooling

What it refuses to do

• Call a pattern's silence a clean bill. Every report states what it CHECKED and what it did NOT, with a denominator. A fixed list of literal spellings reporting "no wrong price found" is a certificate issued by a check that never asked the question.
• Resolve a judgement with a cutoff. A confidence grade, a cosine score and a vote tally are magnitudes a model produced; the cutoff would be ours. Splits go to an adjudicator that reads the code. Unanimity is not a threshold — it is the absence of disagreement.
• Decide meaning with a regex. Regex parses known shapes. It never classifies.

Depends on

• [symgrep-codesearch] (imported as symgrep) — symbol extraction and search-by-meaning. Deliberately not vendored: a second AST layer here would be the exact defect this toolkit detects. Pulls veccore transitively.
• [llm-spendguard] (imported as spendguard) — the gate. Every call is estimated first, caged, recorded and attributable. The distribution is llm-spendguard; the unrelated spendguard on PyPI is a different project.

Tests

python tests/test_toolkit_holds_itself_to_it.py

A review toolkit that violates its own doctrine is worse than none — it certifies the defect it contains. Every assertion in that file is a rule this package states publicly.

Docs

• INTENT.md — the North Star: what the toolkit is for and the invariants it commits to (the intent-alignment axis reads it to judge the repo against its own purpose).
• docs/ARCHITECTURE.md — the HOW: the LOCATE→JUDGE→REFUTE→ACCOUNT pipeline, the two delivery surfaces, and the concurrency architecture (parallel_judge + the pinned-vendor matrix fans; spendguard owns concurrency).
• docs/SPENDGUARD_INTEGRATION.md — the gate contract: matrix fans, metered_only, attribution, and how to run a deep wave review.
• docs/EFFICIENCY.md — the measured cost/latency levers (one retired because measurement falsified it).
• DOCTRINE_CATALOG.md — every doctrine, by domain (regenerate with honestreview doctrines --out DOCTRINE_CATALOG.md); docs/DOCTRINE_ROADMAP.md — the append-only design log; docs/VOCABULARY.md — what "square" and "cubed" mean.