VerifyDoc

The trust layer for document → structured-JSON extraction. Wrap any extractor — get back JSON where every field carries a calibrated confidence, a source grounding (page + bbox / char span), and an accept/review decision tuned to your error budget.

Every other parser tells you what it read; VerifyDoc tells you which values to trust.

▶ Try the live demo or 🤗 the Hugging Face Space — no install: paste a receipt or upload a PDF and watch fields get accepted or routed to review. (Local extraction; for the Claude model, paste your own API key — it's used only for that request and never stored.)

🔒 Private by default — your documents never leave your machine. Every extractor can run fully local and offline (RapidOCR, PaddleOCR, dots.ocr, Docling, or a local HF VLM); hosted API models are opt-in and comparison-only. Self-host the whole review app + API on your own infra, or call it from a web app or a WhatsApp / Telegram bot — the operator controls the data end to end.

Above: a real pipeline run (scripts/make_demo_gif.py). The extractor returned $1,432.50; the page says $1,234.50. Grounding support drops to 0.78, the field misses the accept threshold, and the reviewer is pointed at the exact source region. The other three fields are auto-accepted.

The problem

Modern document parsers read pages at 96%+ benchmark accuracy — and still emit fluent, plausible, silently-wrong values ($42.50 → $45.20) with no reliable per-field signal telling you which values to trust. Commercial APIs (Box, Azure, Textract) sell field-level confidence as a closed feature. No popular open-source parser leads with it. (full USP audit)

VerifyDoc doesn't compete with the parsers — it layers on top of any of them:

document + schema ─► ingest ─► extractor adapter ─► confidence ─► calibration
                                (any model)          signals       (fit on cal split)
                     ─► grounding ─► abstention policy ─► verified JSON + review UI
                        (bbox/span)   (target risk α)

At a chosen operating point, VerifyDoc auto-accepts as many fields as possible while holding the error rate among accepted fields below your target (e.g. ≤ 2%) — everything else is routed to review with its source location attached, so a human verifies in seconds instead of eyeballing every field.

Quickstart

pip install verifydoc                 # core (text pipelines + eval harness)
pip install 'verifydoc[pdf]'          # + PDF/image ingestion
uvx verifydoc extract doc.pdf --schema schema.json   # zero-install run (uv)
pipx install verifydoc                # isolated CLI install
docker run -p 8000:8000 ghcr.io/bhaskargurram-ai/verifydoc   # self-hosted API + web UI

from verifydoc import verify

# ready-to-run sample lives in examples/
result = verify("examples/invoice.txt", schema="examples/invoice_schema.json")
for f in result.fields:
    print(f"{f.path:12} = {f.value!r:24} conf={f.confidence:.2f} {f.decision}")
    if f.grounding:
        print(f"             └─ page {f.grounding.page}, bbox {f.grounding.bbox}")

verifydoc extract examples/invoice.txt --schema examples/invoice_schema.json --threshold 0.8
verifydoc batch ./invoices --schema schema.json -o out/   # whole folder → one JSON/doc + summary.json
streamlit run ui/streamlit_app.py     # review UI: green/red fields + click-through to source

See examples/ for the full runnable walk-through.

For AI agents (MCP)

Give any MCP-capable agent (Claude Desktop, IDEs, custom agents) a trust layer for reading documents — so it acts on confident, grounded fields and escalates the rest instead of hallucinating forward:

pip install 'verifydoc[mcp]'
verifydoc-mcp     # stdio MCP server exposing verify_extraction()

Every field the agent extracts comes back with confidence + grounding + accept/review. See docs/MCP.md for the one-line client config.

Schemas are plain JSON Schema, with each leaf optionally declaring how it is scored (the executable-schema pattern):

{
  "type": "object",
  "properties": {
    "invoice_id": {"type": "string"},
    "vendor":     {"type": "string", "x-scoring": "semantic"},
    "total":      {"type": "number", "x-numeric-tol": 0.01}
  }
}

Integrations

VerifyDoc is a drop-in trust layer — it wraps whatever you already use. No framework is a hard dependency; the framework integrations work by duck typing (any pydantic.BaseModel, any str -> dict callable).

See docs/QUICKSTART_INTEGRATIONS.md for copy-paste snippets and examples/ for runnable end-to-end scripts.

How VerifyDoc compares

VerifyDoc doesn't replace your parser — it adds the trust layer none of them ship:

_{*layout parsers emit boxes but no per-field correctness signal. Full audit: docs/USP.md.}

What's inside

Every metric implements the exact definition in PROJECT.md §5 with a hand-computed numeric regression test (382 tests, eval/ coverage 97%).

Results on real documents

Two independent real OCR extractors (RapidOCR and PaddleOCR) on real CORD receipts and FUNSD forms, scored by the harness (regenerate the full tables locally with make results):

• Grounding is a real trust signal: grounded fields are 84–85% correct vs ~1% for ungrounded (gap ≈ +0.84; box accuracy @IoU 0.5 = 0.75–0.78).
• The abstention layer is honest: with a weak field-extractor the base error rate is high, so conformal abstention at a 2–5% budget correctly refuses to auto-accept — you report selective risk, not headline accuracy.
• The synthetic slice (strong extractor) shows the other end: Coverage@2% ≈ 1.0.

The thesis holds on real data: grounding + a learned fusion rank errors; self-reported and single-sample-consensus confidence do not.

The benchmark

make results     # regenerates every benchmark table/figure from configs/ (local output)

The harness runs signals × calibrators × the full metric suite with a document-level calibration split (disjointness asserted in code), bootstrap CIs, and a conformal-guarantee row. It ships a deterministic synthetic slice (runs in CI) plus CORD and FUNSD loaders with gold source boxes; extractor: dispatches to any adapter (rapidocr, paddleocr-vl, …) and dataset: to any slice. See the GPU runbook to reproduce the real-model rows. Core claims (grounding beats verbalized; conformal holds its guarantee) are also CI-enforced as unit tests, not just stated.

Why not just use the parser's own score?

Because it doesn't exist (Docling/MinerU/Marker), or it's a raw recognition score that was never calibrated against field-level correctness (PaddleOCR/dots.ocr). See docs/USP.md for the audit, and the reliability diagrams make results produces for what "calibrated" actually buys you.

Roadmap

• v0.1 — library + CLI + harness + synthetic benchmark slice + UI
• v0.2 — CORD + FUNSD real slices with gold boxes; learned combiner; 1000× faster grounder
• v0.3 — real-model results (RapidOCR + PaddleOCR on CORD/FUNSD)
• v0.4 — vendor-neutral API-VLM extractor (OpenAI/Anthropic) with k-sample consensus; compilable paper with auto-generated tables
• v0.5 — novel method (grounding-conditioned conformal, +0.50 coverage at fixed risk) + MCP server (agent trust layer) + real frontier-VLM results
• v0.6 — method validated on real data at scale (FUNSD 24%→71% coverage at 2% risk); inter-annotator-agreement tooling (verifydoc iaa); numeric-aware grounding
• v0.10 — live hosted demo + 🤗 Hugging Face Space; verifydoc batch <dir>; adaptive-k consensus; entailment-based grounding (NLI); array-leaf alignment; LangGraph review agent
• dots.ocr via vllm; SROIE / DocILE / XFUND slices; human-labeled correctness at scale
• Paper submission (contributions welcome)

Documentation

• Framework integrations — drop-in trust layer for Instructor, Pydantic-AI, Outlines, LangChain
• How it works — the pipeline, the abstention idea, why grounding is a trust signal
• Related work & positioning — how VerifyDoc compares to Beyond Logprobs, Cleanlab TLM, conformal factuality, and the commercial IDP stack
• MCP for agents — client config + the registry publish runbook
• GPU runbook — reproduce the real-extractor rows
• USP audit · Full spec

Development

git clone https://github.com/bhaskargurram-ai/verifydoc && cd verifydoc
uv venv .venv && uv pip install -e ".[dev]"
make test lint typecheck     # all green before any PR (CI enforces)
make results                 # regenerate benchmark tables + LaTeX
make paper                   # compile the paper (needs a LaTeX toolchain)

Contributions welcome — see the issues tagged good-first-issue. All model-specific code goes in verifydoc/adapters/; a new extractor is one file.

Citation

@software{verifydoc2026,
  author = {Gurram, Bhaskar},
  title  = {VerifyDoc: Calibrated, Abstaining, Grounded Document Extraction},
  year   = {2026},
  url    = {https://github.com/bhaskargurram-ai/verifydoc}
}

Apache-2.0.