AnchorCite

Anchor AI-generated claims to the exact supporting sentences in their cited sources, with character offsets and entailment scores.

Extracted from the citation Finalizer of Bahith, a deep research API where every generated claim links to the exact supporting sentence in its source.

---

Overview

AI-generated answers frequently cite source documents at the document or URL level, leaving users to manually locate the specific sentence that supports a given claim. AnchorCite pinpoints the exact sentence(s) within source documents that support each claim, returns start and end character offsets (allowing developers to source_text[start:end] == quote) along with a support probability score.

Key Features

• Sentence-Level Precision: Resolves claims directly to character offsets (source_text[start:end] == quote) in source documents.
• Hybrid Retrieval: Combines BM25-Okapi term matching with bge-small dense cosine similarity via Reciprocal Rank Fusion (RRF) to rank candidate sentences without normalization skew.
• Cross-Encoder Verification: Scores candidate sentences using NLI cross-encoders (defaulting to MiniCheck DeBERTa-v3).
• Verbatim Fast-Path: Instantly anchors exact or near-exact quotes without running model inference.
• Rule-Based Atomization: Decomposes compound multi-clause claims so each distinct fact grounds against its own supporting sentence.
• Dirty-Input Handling: Bibliography fragments, DOI headers, and institutional or author affiliation blocks are filtered out during anchoring. A separate normalize_extracted_text helper repairs glued sentence boundaries from PDF and HTML extraction. You can apply it yourself before anchoring, since offsets are always relative to the text you pass in.
• Lightweight Core: Sentence splitting, BM25, RRF, and text cleaning are implemented from scratch. The core install depends only on pydantic, and torch, sentence-transformers, and numpy are only installed with the optional model extra.

---

Installation

Install the lightweight core library (includes standard BM25 and lexical scoring):

pip install anchorcite

To enable neural entailment verification with sentence-transformers and torch, install the model extra:

pip install "anchorcite[model]"

While the version is below 1.0, minor version updates may change the API or the verification behavior. Pin to anchorcite~=0.1.0 if you need stability.

---

Quickstart

1. Anchor a Single Claim (anchor_claim)

from anchorcite import anchor_claim

source_text = (
    "Solid-state cells cycled at 45 C lost 18 percent of nominal capacity. "
    "Identical cells held at 25 C lost only 6 percent over the same period. "
    "Dendrite growth scaled superlinearly with current density above 3 mA/cm2."
)
claim = "Cells cycled at 45 C lost 18 percent of nominal capacity. "

anchor = anchor_claim(claim, source_text)

if anchor:
    print(f"Verified Quote: {anchor.claim}")
    print(f"Offsets: [{anchor.start_offset}:{anchor.end_offset}]")
    print(f"Support Score: {anchor.score:.3f}")

Output (core install):

Verified Quote: Solid-state cells cycled at 45 C lost 18 percent of nominal capacity.
Offsets: [0:69]
Support Score: 1.000

2. Verify an Answer with Citation Tags (verify_answer)

from anchorcite import verify_answer

answer = (
    "Accelerated aging tests show <cite id=\"s1\">18 percent capacity loss after "
    "800 cycles</cite> at elevated temperature. The same cells also showed "
    "<cite id=\"s2\">dendrite growth that scaled with current density</cite>."
)
sources = {
    "s1": "Solid-state cells cycled at 45 C lost 18 percent of nominal capacity.",
    "s2": "Dendrite growth scaled superlinearly with current density above 3 mA/cm2 in the same cell chemistry.",
}

report = verify_answer(answer, sources)

print(f"Verified Fraction: {report.verified_fraction:.1%}")
print("\n" + report.to_markdown())

Output (core install):

Verified Fraction: 100.0%

 | Claim | Verified | Score | Supporting Quote | 
 | :------ | :------ | :------ | :------ | 
 | 18 percent capacity loss after 800 cycles | ✅ Yes | 0.833 | Solid-state cells cycled at 45 C lost 18 percent of nominal capacity. | 
 | dendrite growth that scaled with current density | ✅ Yes | 1.000 | Dendrite growth scaled superlinearly with current density above 3 mA/cm2 in the same cell chemistry. | 

A note on the two install tiers

The outputs above come from the core install, which has no ML dependencies. Scores there are a lexical fallback: 1.000 means the verbatim fast path matched an exact span, and 0.833 is content-token overlap. Overlap is a ratio of small integers, so it is coarse by nature and works best when the claim reuses the source's wording.

Installing anchorcite[model] replaces that fallback with the MiniCheck DeBERTa-v3 cross-encoder. Scores become entailment probabilities, so a paraphrase that shares few words with its source can still score highly, and an unsupported claim that happens to reuse the source's vocabulary can score low. Offsets and the fast path behave identically in both tiers.

3. Command-Line Interface (CLI)

Verify answers directly from the terminal using answer files and a sources directory or JSON file:

anchorcite verify -a answer.txt -s ./sources_dir --format markdown

Pass --model to enable neural entailment scoring:

anchorcite verify -a answer.txt -s ./sources.json --model --output report.json --format json

Pass --fail-under to gate a verification on the verified fraction:

anchorcite verify -a answer.txt -s ./sources.json --fail-under 0.9

The command exits 0 if the verified fraction meets or exceeds --fail-under, 1 when it falls below, and 2 when the run could not complete at all, so a CI job can distinguish a poorly grounded answer from a broken invocation. The default of 0.0 never fails.

Both quickstart examples above, as well as an example with a claim the source does not support, are in examples/quickstart.py.

---

Pipeline Architecture

Sentence Filtering: Source sentences that look like bibliography fragments, DOI headers, or institutional or author affiliation lists are dropped before retrieval, so claims can't ground against reference-list noise. Text normalization is deliberately not applied here because it would change string offsets, so normalize_extracted_text is exposed as a preprocessing step for the caller to run first.

Verbatim Fast-Path: Spans of 6 or more words with high lexical similarity (fuzzy ratio >= 0.90) anchor immediately at score 1.0 without running neural compute.

Hybrid Retrieval: Candidate sentences are retrieved by fusing Okapi BM25 scores and dense bi-encoder embeddings (BAAI/bge-small-en-v1.5) via Reciprocal Rank Fusion (RRF). RRF is scale-free, eliminating manual score normalization.

Entailment Verification: Candidate sentences are scored against the claim using lytang/MiniCheck-DeBERTa-v3-Large.

Re-Ranking & Diversity: Candidates are re-ranked to prefer sentences not already used by another claim in the same answer, so a single quotable sentence does not absorb every citation. Per-source sentence spans, embeddings, and BM25 indexes are memoized in an LRU cache so repeated claims against one source only pay the setup cost once.

---

Calibration Note

Support thresholds are properties of a scorer's score distribution, not of the task, so each scorer carries its own. Both values were measured on the WiCE dev split using benchmarks/run_benchmark.py, choosing a flat region of the precision and recall curve rather than the point that simply maximises f1:

scorer	threshold	window	precision	recall
LexicalScorer	0.40	0	0.843	0.783
EntailmentScorer	0.15	1	0.919	0.704

Pass threshold= to verify_answer or CitationAnchorer to override either. A scorer of your own that declares neither attribute falls back to DEFAULT_SUPPORT_THRESHOLD.

These numbers come from Wikipedia claims and their cited sources. You should recalibrate threshold and window on your own domain for a production deployment.

---

Benchmark

benchmarks/run_benchmark.py measures AnchorCite on WiCE (Kamoi et al., EMNLP 2023), which pairs a Wikipedia claim with the source article it cites and records which sentences a human annotator found supporting. The dataset is fetched on demand into a gitignored cache rather than committed, since its annotations are ODC-BY and its evidence text is Wikipedia CC-BY-SA plus Common Crawl.

python benchmarks/run_benchmark.py

Measured on the 776 scored subclaims of the dev split, skipping 173 partially supported ones. Each tier runs at its own defaults:

	core install	anchorcite[model]
source_text[start:end] == quote	492 / 492	406 / 406
precision	0.843	0.919
recall	0.783	0.704
f1	0.812	0.797
unsupported claims rejected	169 / 246 (68.7%)	213 / 246 (86.6%)
quote touches a gold sentence	75.2%	81.5%
mean span overlap (jaccard)	0.389	0.418
elapsed	12s	47m, using RTX 2060

Don't read too much into f1. Answering "supported" for everything scores 0.812 as well, because 68% of WiCE claims are supported. AnchorCite's core and model tiers reject 69% and 87%, respectively, of unsupported claims and return exact spans for supported ones, which f1 can't capture. A wrongly certified claim costs more than an unsupported one.

The model tier is tuned for precision while maintaining high recall. At 0.05 threshold it hits 0.828 recall and 0.856 precision, beating the core install on both, but 0.15 keeps the wrongly-certified rate lower (8% rather than 14%).

---

Scope & Non-Goals

AnchorCite focuses strictly on sentence-level claim verification and attribution. It does not perform:

Cross-source document retrieval or web crawling. LLM answer generation. Arbitrary claim extraction without target citation tags.

---

License & Attribution

Distributed under the MIT License.

This library builds upon research and open models from the community:

MiniCheck: Tang, Laban, and Durrett (2024). MiniCheck: Efficient Fact-Checking for LLMs. BGE Embeddings: BAAI (BAAI/bge-small-en-v1.5).