citeverify

Check whether the references in a bibliography actually exist.

Fabricated citations are now a routine failure mode. LLM-drafted reference lists invent plausible-looking papers; copy-paste drift corrupts real ones. citeverify cross-checks every reference against OpenAlex, Crossref and arXiv, and tells you what it could and could not confirm.

pip install citeverify
citeverify check references.txt

Checked 6 references
----------------------------------------------------
  verified              4   ( 66.7%)
  year mismatch         1   ( 16.7%)
  NOT FOUND             1   ( 16.7%)

2 reference(s) need attention:

  [NOT FOUND] Nonexistent, A. (2023). Quantum entanglement in transformer attention heads.
      closest: Transformer quantum state: A multipurpose model for quantum many-body problems (2023)
  [year mismatch] Smith, J. (1897). Attention is all you need. NeurIPS.
      closest: Attention Is All You Need (2017)

Why the verdicts are split the way they are

Most of the design effort went into not crying wolf. A tool whose output reads as an integrity signal has to be careful about what it asserts.

Three rules follow from this, and they are enforced by tests:

1. A transport failure is never a verdict. If every index errors you get error, never unverified. Reporting a real paper as missing because DNS failed is the worst thing this tool could do.
2. Missing author data never counts against a reference. Standards, datasets and technical reports routinely list no authors. Absence of evidence is not evidence of fabrication.
3. Wrong metadata is distinguished from invention. A real paper cited with a bad year resolves to the right work and is reported as a year error. A typo and a hallucination need different fixes.

Usage

CLI

citeverify check refs.txt                      # human-readable report
citeverify check refs.txt --json out.json      # machine-readable results
citeverify check refs.txt --cache .cv.json     # reruns become free
citeverify check refs.txt --strict             # exit 1 if anything suspicious (CI)
citeverify parse refs.txt                      # parse only, no network
citeverify check - < refs.txt                  # read stdin

Give --mailto you@example.org to enter the OpenAlex/Crossref polite pool, which gets you faster and more reliable service. Thresholds are tunable: --verified-threshold, --partial-threshold, --year-tolerance.

Exit codes: 0 ran cleanly · 1 suspicious references found under --strict (error results never trigger this) · 2 usage or input error.

Python

from citeverify import parse_references, verify_all, summarize

refs = parse_references(open("references.txt").read())
results = verify_all(refs, cache=Cache(".cv.json"))

print(summarize(results))                       # {'verified': 4, 'unverified': 1, ...}
for r in results:
    if r.suspicious:
        print(r.status, "--", r.citation["raw"])

Already have structured data? Skip the parser. verify() and verify_all() accept plain dicts with title / authors / year / doi / arxiv_id.

Air-gapped and restricted networks

All network access goes through a single injectable callable. Nothing else in the package performs I/O:

from citeverify import verify, OpenAlexSource

def my_fetcher(url, headers):
    return my_corporate_proxy.get(url, headers=headers).text

verify(citation, sources=[OpenAlexSource(fetcher=my_fetcher)])

Point it at an on-prem OpenAlex mirror, a proxy, or a cached snapshot. The entire test suite runs through this seam, so it is a supported path and not an afterthought.

Install

pip install citeverify

Zero hard dependencies. Standard library only. This is deliberate: the people who most need citation checking (institutional review offices, air-gapped labs) are the least able to approve a dependency tree.

Requires Python 3.9+.

Limits, and please read them

• The parser is heuristic. Reference lists are a hundred house styles wearing a trenchcoat. Entries parsed by segmentation are marked parse_confidence: "low" and flagged in notes. Feed structured data when you have it.
• Matching is lexical. Title similarity is token overlap with a substring escape hatch. It does not understand meaning.
• Coverage gaps are real. Books, theses, standards, and much non-English and pre-1970 work are thinly indexed. unverified on those usually means the work is not indexed. It does not mean the work is fake.
• It does not judge whether a citation is appropriate, or whether the cited work supports the claim attached to it. It answers exactly one question: does a record matching this reference exist?

Treat the output as a triage queue for a human, never as a verdict.

Provenance

The matching logic was developed and hardened while verifying a 1,473-reference corpus across ~101 papers. Every one was driven to a resolved status. This is extracted from tooling that had to work at that scale.

Development

pip install -e ".[dev]"
pytest            # 76 tests, all offline. No network required.

Licence

MIT.