llmground

Your schema validated. The value was still made up.

Structured output libraries guarantee the shape of extracted data. None of them check whether the values are actually in the document. llmground runs after extraction and tells you which fields are real.

from llmground import verify

report = verify(extracted, source_document)

if report.has_hallucinations:
    route_to_human_review(report.ungrounded)

No API calls. No model. No cost. Zero dependencies.

The bug this catches

extracted = {
    "vendor": "ACME Manufacturing Ltd.",
    "total": 2846.25,          # the invoice says 2,486.25
}

Right types. Passes any JSON schema. Passes Pydantic. The number is a transposition the model invented, and it goes straight into your accounting system. Shape validation cannot see this, because there is nothing wrong with the shape.

GROUNDED    vendor = 'ACME Manufacturing Ltd.'
            found: ACME Manufacturing Ltd. 14 Sundar Industrial Estate...
UNGROUNDED  total = 2846.25
            nearest number in source is '$2,486.25'

6/8 fields grounded (75%), 0 weak, 2 ungrounded

Install

pip install llmground

It works with what you already use

llmground doesn't replace Instructor, Outlines, or PydanticAI. It runs after them. Pydantic models, dataclasses, and plain dicts all work, and Pydantic is optional rather than required.

import instructor
from llmground import verify

invoice = client.chat.completions.create(response_model=Invoice, ...)

report = verify(invoice, source_pdf_text)
if report.grounding_rate < 0.9:
    escalate(report)

Why matching values is harder than it looks

The naive version of this check is str(value) in source, and it produces so many false alarms that people switch it off within a day. Real documents don't write values the way models return them.

Every one of these is handled deterministically, with no model in the loop.

Verdicts

Fuzzy matches are weak, not grounded, by default. A 90%-similar string is a different string, and on a financial or medical field that difference is the entire problem. Opt in with accept_fuzzy=True when you're handling OCR noise and know what you're trading away.

Skipped fields are excluded from the grounding rate. Counting booleans as grounded would inflate the score and hide real problems.

Spans, for human review

Every match reports character offsets into the source, so you can highlight the evidence rather than just flagging a field.

for r in report.grounded:
    span = r.match.spans[0]
    highlight(source, span.start, span.end)

That's the difference between telling a reviewer "this field is suspicious" and showing them why.

CLI

llmground extracted.json --source invoice.txt --fail-under 0.9

Exits non-zero below the threshold, so it drops into CI as a quality gate.

Options

verify(
    extracted,
    source,
    accept_fuzzy=False,        # count fuzzy matches as grounded
    fuzzy_threshold=0.85,      # similarity floor for reporting a fuzzy match
    number_tolerance=0.01,     # allowed numeric difference
    day_first=True,            # how to read 03/04/2026
    ignore=["metadata.run_id"],  # skip computed fields
)

Limitations

• Extractive only. It verifies values that should appear in the source. A field that's legitimately inferred rather than copied (a category label, a computed total) will read as ungrounded. Use ignore for those.
• Ambiguous dates are not guessed. 03/04/2026 returns no match unless you pass day_first, because a silently wrong date is worse than an unverified one.
• Grounded is not correct. It confirms a value appears in the document, not that the model pulled it from the right place. A total that matches a different line item's amount will pass.
• Text sources only. PDFs and images need to be through OCR first.

Development

pip install -e ".[dev]"
pytest

MIT.