mcp-citation-checker

An MCP server that verifies whether a claim is actually supported by the source text at a given citation — independent of what the calling LLM asserts.

Why this exists

LLM agents that cite sources can still get the citation wrong: paraphrasing loosely, citing the wrong page, or asserting a source says something it doesn't. check_citation closes that gap by independently fetching the real text at the claimed citation and asking a separate LLM call to judge, strictly from that text, whether the claim holds up. The agent's own paraphrase is never trusted as evidence — only the independently-fetched source is.

Design: pluggable backends

Looking up "the real text at citation X" depends entirely on where your documents actually live — Weaviate, Postgres, a flat file, something else. Rather than hardcoding one storage system, this package defines a small interface any backend can implement:

class SourceLookup(Protocol):
    def fetch(self, pdf_name: str, page_number: int) -> list[dict]:
        ...

weaviate is included out of the box as a working, tested implementation (bundled as an optional extra, so installing the base package doesn't force a Weaviate dependency on anyone who doesn't need it). Writing your own backend for another storage system just means implementing that one fetch method — see Writing your own backend below.

Install

pip install "mcp-citation-checker[weaviate]"

(Omit [weaviate] if you're supplying your own backend instead.)

Configuration

Set via environment variables when the server starts:

Usage

Run directly:

mcp-citation-checker

Or wire it into an MCP client config (e.g. Claude Desktop, or your own agent) like any other stdio MCP server:

{
  "mcpServers": {
    "citation-checker": {
      "command": "mcp-citation-checker",
      "env": {
        "ANTHROPIC_API_KEY": "...",
        "WEAVIATE_MODE": "cloud",
        "WEAVIATE_CLOUD_URL": "...",
        "WEAVIATE_API_KEY": "..."
      }
    }
  }
}

The server exposes one tool, check_citation(claim, pdf_name, pdf_page), returning "<label>: <one-sentence reasoning>" where <label> is one of supported, contradicted, or not_mentioned.

Writing your own backend

The verification logic (mcp_citation_checker.checker.check_citation) has no storage-specific coupling at all — it only needs claim and a list of chunk dicts (pdf_name/page_number/text). Any backend just needs to implement the SourceLookup shape:

class MyBackend:
    def fetch(self, pdf_name: str, page_number: int) -> list[dict]:
        # return [{"pdf_name": ..., "page_number": ..., "text": ...}, ...]
        ...

No inheritance required — SourceLookup is a typing.Protocol, so any object with a matching fetch method satisfies it.

Two ways to actually use a custom backend:

• Bypass the bundled server entirely: call checker.check_citation(claim, my_backend.fetch(pdf_name, page)) directly from your own code/MCP server — this works today, no changes to this package needed.
• Wire it into this package's own server: the current version only registers a weaviate backend in _get_backend() (src/mcp_citation_checker/__init__.py) — using a different backend through the bundled mcp-citation-checker command currently means adding a branch there yourself (a small, welcome PR).

Development

pip install -e ".[weaviate,dev]"
pytest

License

MIT — see LICENSE.