scholarcheck

Stop hallucinated citations. Verify any reference against real metadata — from the command line, with zero dependencies.

_{The figure above is generated by docs/three-states_figure.py — pip install sciglyph and run it to reproduce docs/three-states.png byte for byte.}

Language models invent plausible-looking papers: right-sounding title, plausible authors, a DOI that resolves to nothing. scholarcheck answers one question honestly — does this paper actually exist? — by querying OpenAlex, Semantic Scholar, Crossref and arXiv directly.

$ scholarcheck verify "Deep Residual Learning for Image Recognition"
MATCH (high confidence)   [query term coverage = 100%]
Deep Residual Learning for Image Recognition  (2016, conference-paper; cited=226875)  doi:10.1109/cvpr.2016.90
    Kaiming He, Xiangyu Zhang, Shaoqing Ren et al.

$ scholarcheck verify "Quantum Topological Radiomics for Zebra Diagnosis in Martian Cohorts"
NOT FOUND in any of the four sources -> this citation is very likely hallucinated

Why not just ask an AI assistant?

Because an assistant answers from memory, and memory is exactly what fails here. Three design choices make this different:

1. It says "I could not check" instead of "it is fake." A verifier that reports a network outage as hallucinated is worse than no verifier. scholarcheck tracks every failed request and distinguishes the two:

$ scholarcheck verify "Attention Is All You Need"     # with the network down
INCONCLUSIVE - could not query the sources, so nothing can be said about: Attention Is All You Need
  Could not reach: api.openalex.org: curl: (7) Connection refused
  (no proxy set; if your network needs one, set SCHOLARCHECK_PROXY)

It also knows which sources matter: Semantic Scholar rate-limits aggressively without an API key, so its failure never turns a real answer into "inconclusive" — only the primary sources do.

2. It refuses to guess. Ask for BibTeX from a slightly-wrong title and most tools hand back the nearest hit. Silently citing the wrong paper is worse than citing none, so a weak match returns the candidate and stops:

$ scholarcheck bibtex "Deep Residual Learning for Image Recognition in Medicine"
No confident match (best term coverage only 62%). Refusing to emit a possibly wrong entry.
Closest candidate:
  Deep Residual Learning for Image Recognition  (2016, CVPR)  doi:10.1109/CVPR.2016.90
-> If that is the paper, re-run with its DOI: scholarcheck bibtex "<DOI>".

The same refusal applies when the sources themselves are unavailable, which is when a wrong entry is most likely — the "best" match would then be whichever paper happened to be reachable:

$ scholarcheck bibtex "Deep Residual Learning for Image Recognition in Medicine"
INCONCLUSIVE - a primary source could not be reached, so no entry is emitted for: ...
  Could not reach: api.openalex.org: HTTP 429
  (the partial search's best candidate was 50% coverage - not enough to stand on
   while sources are down)

3. An identifier is resolved, not searched. verify "arXiv:1906.08253" looks the identifier up directly. Feeding it to a title matcher would return whatever paper happens to share those digits and then score it as a mismatch — which reads as "this citation is fake" when the truth is that the query was never looked up properly.

4. Recency is a separate command, on purpose. Relevance ranking systematically favours highly-cited older work, which is exactly wrong when you are checking whether someone just published your idea. latest filters by recency as well as relevance.

Install

pip install scholarcheck

No dependencies. Standard library plus curl — a fresh virtualenv gains exactly one package and nothing else. Nothing to break, nothing to audit, and no API key: every source it queries is open.

Commands

Add --json to any command for structured output, -n for the number of results, --since YYYY to bound the year.

Use as a library

from scholarcheck import verify_citation, get_bibtex, NET_ERRORS

paper, confidence = verify_citation("Attention Is All You Need")
if paper is None and NET_ERRORS:
    ...          # could not check — not evidence of anything
elif confidence >= 0.75:
    print(get_bibtex(paper["doi"]))

Configuration

All optional:

Proxy behaviour is decided solely by SCHOLARCHECK_PROXY. Inherited http_proxy / all_proxy variables are stripped before each request, so the tool behaves the same on every machine.

What it can and cannot tell you

A match confirms the paper exists — not that the metadata you have is right. Bibliographic databases often hold several records for one work: a preprint, a conference version, a publisher deposit. verify returns whichever record matched best, so the year and venue you see may belong to a different record than the one you meant to cite. Check them; the DOI is the reliable part.

"NOT FOUND" is strong evidence, not proof. Very new work, non-English venues and some book chapters are indexed poorly. When it matters, run search with looser keywords before concluding a reference is invented.

Notes from real use

• Feed focused keywords, not whole sentences. A long claim drags in off-topic papers; two or three precise terms work far better.
• search favours highly-cited older work. That is what relevance ranking does. Use latest when the question is "has this been done recently?"
• A title-only judgement is not a prior-art check. For the closest candidates, fetch the PDF and read it.

Who maintains this

Guo Cheng, University of Chinese Academy of Sciences — medical imaging and machine learning methods. This tool came out of checking my own citations before submitting, after watching a language model hand me three papers that did not exist.

Corrections, bug reports and feature requests all go to Issues. Please open one rather than emailing: a public answer helps whoever hits the same thing next, and it is searchable.

License

MIT © Guo Cheng

关于那行 star 提示

跑命令时，scholarcheck 会在第 5 次和第 25 次往 stderr 写一行，提一句这个仓库在哪。一辈子只有这两次，此外再不出声。

它不会出现在：管道或重定向里（stderr 不是终端就直接返回，连计数文件都不建）、CI 环境里（CI / GITHUB_ACTIONS）。它写的是 stderr 而非 stdout，所以不会污染你的数据输出；它包在 try/finally 里且吞掉自身所有异常，不会改变退出码，也不会影响结果。

永久关掉：

export SCHOLARCHECK_NO_NUDGE=1

计数存在 $XDG_STATE_HOME/scholarcheck/usage.json（默认 ~/.local/state/scholarcheck/usage.json），删掉即重置。