Reviewer Zero

An open-source MCP server that reviews your ML paper before submission the way a strict PI would: format and anonymization, every reference, prior work for each claim, a methodology checklist and a writing check, with every quote checked against your PDF. It runs inside Claude Code or Claude Desktop, on your own Claude plan. It never gives a score, a rating or an accept/reject verdict, and it never rewrites your text.

MIT licensed. Package reviewer-zero (import name reviewer).

Install

You need uv and a free index key. Get a free key at https://tryreviewerzero.com. Questions: hello@tryreviewerzero.com. The key meters use of the hosted search index; it is not a payment method.

Claude Code

claude mcp add reviewer-zero -e REVIEWER_ZERO_INDEX_KEY=<your key> -- uvx reviewer-zero
# the review workflow (the skill), so Claude knows the order and the rules:
git clone https://github.com/rishab-ghosh/reviewer-zero
mkdir -p ~/.claude/skills && cp -r reviewer-zero/skills/reviewer-zero ~/.claude/skills/

Then ask: "Review my paper at ~/papers/draft.pdf for ICLR 2027."

Claude Desktop: add to claude_desktop_config.json (Settings → Developer → Edit Config):

{
  "mcpServers": {
    "reviewer-zero": {
      "command": "uvx",
      "args": ["reviewer-zero"],
      "env": { "REVIEWER_ZERO_INDEX_KEY": "<your key>" }
    }
  }
}

and add skills/reviewer-zero as a skill (zip the folder and upload it in Claude's skills settings), or paste skills/reviewer-zero/SKILL.md into a project's instructions.

Optional, recommended: GROBID for the best reference parsing (and required by review_paper). It runs locally; your PDF never leaves your machine for it:

docker run --rm -p 8070:8070 grobid/grobid:0.9.1-crf

Without it, check_citations reads the reference list from the PDF's text layer and resolves about 72% as many references (see "Measured" below).

Two ways to review

The default: the skill, on your Claude plan, no API key. Claude reads your paper and writes the review following skills/reviewer-zero/SKILL.md; the tools do retrieval and the checks that need code. Every quote goes through verify_quotes before Claude may use it.

The measured pipeline: review_paper, on your own ANTHROPIC_API_KEY. Our prompts and models, the pipeline we measured on the dev set. It needs a local GROBID and the key in the server's environment (-e ANTHROPIC_API_KEY=...). It always shows an estimate and a hard spending limit first (confirm=false spends nothing) and runs only after you agree. Two novelty settings:

Recall is on the same 50 dev cases (74 prior works named by real reviewers); times and costs are one run each on our 4-page demo paper (docs/demo), so a long paper takes longer.

Timeouts and the cheap resume. Claude Code stops waiting for a tool after its MCP tool timeout; start it with a longer one for review_paper:

MCP_TOOL_TIMEOUT=900000 claude

If a call still times out, call review_paper again on the same PDF: finished steps are served from the local cache (~/.cache/reviewer-zero) and only the unfinished ones are paid for.

Tools

Quota. A key allows 600 units an hour and 5,000 a day. One find_prior_work call costs about 17 units (6–10 searches plus two expansion calls), so about 35 calls an hour.

Privacy

• Reviewer Zero's tools never send your PDF or its text anywhere, except to Anthropic under your own key in review_paper. (In the skill path, Claude itself reads the paper through your Claude client, as with any file you give it.)
• Only search queries, paper keys and the titles, DOIs and arXiv ids of works you cite go to our index; cited works' identifiers may also go to OpenAlex, Crossref and arXiv to resolve them.
• GROBID is used only at a local address (localhost); any other GROBID_URL is ignored.
• No telemetry. The index counts requests per key and stores no query text.
• tests/test_mcp_privacy.py runs every tool on a test paper and fails if any request goes to another host or carries a sentence of the paper's body.

Measured

Dev-set numbers, measured on 2026-10-02 unless noted; details, caveats and suggested wording in docs/MEASURED_CLAIMS.md.

• Prior-work recall@10, measured pipeline (review_paper): 43.2% full, 36.5% lite, on 50 dev cases with 74 reviewer-named prior works.
• Prior-work recall, MCP path (Claude writes the queries from the tool description, then ranks the 40 candidates): 25.7% with Sonnet 5, 27.0% with Opus 5.5 (recall@10, same 50 cases; today's index). The gap to the pipeline is mostly retrieval: the pipeline reranks 150 candidates per claim, the tool returns 40. For the most thorough prior-work search, use review_paper.
• Skill path, end to end (Opus 5.5, 6 dev papers): about 4 minutes per paper; 184 of 188 quotes Claude submitted were verified (it must fix or drop the rest); no score or verdict sentence in any review. Too few papers for a recall number.
• find_prior_work latency: median 5.2 s for 6 queries against the hosted index (16 calls from Austin, TX, 4.7–9.3 s): about 2.4 s of searches, 2.3 s of citation-graph expansion and 0.3 s of metadata, with at most 4 index requests in flight (the per-key limit). On a local copy of the index the same call takes a median 1.6 s.
• check_citations without GROBID: 72.3% as many references resolved as with GROBID (30 dev papers).
• Index: about 1.09 million papers (577k arXiv papers in ML and adjacent fields since 2017, plus works they cite and venue papers not on arXiv), 28.3 million citation edges, updated nightly.

Licence

MIT (LICENSE). Runtime dependencies are permissively licensed: mcp, anthropic, pydantic, pydantic-settings, typer, pyyaml, pdfplumber (MIT); httpx, lxml, pypdf, numpy (BSD); the optional docling extra (MIT). GROBID, run separately as a container, is Apache-2.0.

Development

make setup        # uv sync, pre-commit install
make lint test    # ruff and pytest; tests never touch the network

The pipeline behind review_paper is in src/reviewer/: parse/ (GROBID), claims/, novelty/, methods/, writing/, citations/, format/, meta/ (including the leak filter that keeps scores and verdicts out of every output), review/ (orchestration, budget) and mcp/ (the server and its tools). Prompts are versioned files next to each step.