hallucination-risk-meter

Estimate hallucination risk in LLM answers from uncertainty language, unsupported specifics, citations, and context coverage. Zero runtime dependencies.

Python port of @mukundakatta/hallucination-risk-meter. The JS sibling has the original API; this README sticks to the Python surface.

Install

pip install hallucination-risk-meter

Usage

from hallucination_risk_meter import score

answer = "The company earned $42 million in 2023."
ctx    = "Acme Inc. reported revenue of $40-44 million for fiscal year 2023."

r = score(answer, context=ctx, citations=[{"id": "1"}])

r.score      # float in [0.0, 1.0]
r.signals    # list[str] -- triggered heuristics
r.severity   # 'low' | 'medium' | 'high'

Signals

Each triggered signal contributes a fixed weight to the final score (clamped to [0, 1]):

Severity buckets

• score < 0.34 -> "low"
• 0.34 - 0.66 -> "medium"
• score >= 0.67 -> "high"

Custom signals

Inject precomputed signals from your own classifier:

score("...", signals=["nli_contradiction", "self_consistency_drop"])

Names not in the built-in weight table contribute 0.0 but still appear in r.signals for downstream logging.

API differences from the JS sibling

• Returns a RiskScore dataclass with score, signals, and severity instead of the JS {risk, reasons, likelyHallucinated} object.
• Adds the signals= parameter for injecting upstream-detector signals.
• Adds severity bucketing (low/medium/high) for convenient guardrail thresholds.

See the JS sibling's README for the full design notes.