Reference Verifier

Check that every reference in your paper exists before a reviewer, a venue or arXiv checks for you.

refverify reads your .bib (or your LaTeX paper and the bibliography it uses) and looks each entry up in Crossref, OpenAlex, DBLP, arXiv and PubMed (and Semantic Scholar, if you have a key). It reports references that no registry lists, DOIs that are not registered, and DOIs that belong to a different paper.

$ refverify main.tex

Reference Verifier 0.2.2: 41 references from refs.bib

  37 found · 2 not found · 1 wrong DOI · 1 uncertain

  WRONG DOI  chen2023    Scalable graph transformers for molecules (2023)
                         DOI 10.1016/j.patcog.2021.108045 belongs to a different work: "..."
  NOT FOUND  smith2024   Quantum attention for genome alignment (2024)
                         no matching work in Crossref, OpenAlex, DBLP or arXiv
  ...

Install

pip install reference-verifier

Python 3.10 or newer. To keep the command in an environment of its own: pipx install reference-verifier or uv tool install reference-verifier.

Use

refverify refs.bib                 # every entry in a .bib
refverify main.tex                 # only the keys main.tex cites (follows \input and \bibliography)
refverify main.tex --json out.json # machine-readable results too

Exit status is 1 when a journal or conference reference is not found or has a bad DOI, so refverify can run in a script or CI job. --strict also fails on uncertain matches.

Set your email so Crossref and PubMed can contact you if something goes wrong (their "polite pool"):

export REFVERIFY_MAILTO=you@example.org

OpenAlex searches draw on a daily budget ($0.001 per search; looking a DOI up is free). Without a key everyone on your network shares $0.10 a day; a free key from openalex.org gives you $1 a day of your own, enough for about a thousand searches:

export OPENALEX_API_KEY=...

Semantic Scholar finds reports and preprints the other registries miss, but its anonymous pool is usually rate-limited, so it is used only when you set a key (free on application at semanticscholar.org):

export S2_API_KEY=...

If a registry stops answering during a run (rate limits, a bot check), refverify skips it for the rest of the run and says so; rerun later to include it.

What the results mean

A reference that is not found is not proof of fabrication, and a found one can still have a wrong year, venue or author list. refverify checks existence.

How well it works

Measured with 0.2.0 (0.2.1 changes only how file names are shown, 0.2.2 only how it is packaged) on a held-out benchmark of 246 labelled references that was built separately from the development set and from the earlier held-out set, and scored once, with no changes to the matching rules afterwards (benchmark/heldout-0.2/; sources, licences and labelling rules in HELDOUT.md):

• The real reference it missed is a DOI-less ICLR 2022 paper that only DBLP indexes; DBLP's search turned the tool away during the run (it shows a bot check to scripts, which the tool does not try to get past).
• Real references were drawn at random across computer science, biomedicine, physics, mathematics, social science, the humanities and other fields, written in the messy forms real .bib files use. The fabricated ones come from public compilations of hallucinated citations found in ACL-family papers and ICLR 2026 submissions, so all 54 are computer science.
• "Fabricated" means listed by one of those compilations and found neither in any registry nor by a web search for the exact title. That is strong evidence, not proof.
• 0.1.0 was scored on an earlier held-out set (benchmark/heldout/; its numbers are in the CHANGELOG). The development set the rules were tuned on (342 references) is in benchmark/; its scores are in benchmark/results/summary.md and are not a fair measure. The benchmark is in the repository, not in the package.

Rerun it from a clone of the repository: python benchmark/evaluate.py --data benchmark/heldout-0.2/heldout.jsonl --out benchmark/heldout-0.2/results --heldout (results vary slightly as registries change and become reachable or not).

Privacy and etiquette

Titles, first authors, years and identifiers from your bibliography are sent to the registries listed above. Nothing else leaves your machine. Answers are cached for 30 days in ~/.cache/reference-verifier/, so repeat runs are fast and light on the registries. Requests are rate-limited per registry, including the one request every three seconds that arXiv asks for.

Reference Verifier Pro

This free core checks that each reference exists. Reference Verifier Pro, a paid add-on, also checks each one against the publisher's record and the retraction databases (retractions and corrections, wrong DOIs with the likely correct one, authors, years, venues and pages, duplicates and cite keys, published versions of preprints), writes corrected copies to review and dated audit reports, and checks a whole class's papers at once. A real run, the held-out benchmark and the licences: https://the-aurora-lab.github.io/tools/reference-verifier/

Licence

MIT