triple-check
Small local LLMs are confident and bad at arithmetic. Ask a 3B model how much land a tequila brand needs and it will tell you a 9-litre case holds 36 litres.
triple-check is a pattern (and a small library) for making local-model answers
trustworthy enough to act on:
- Python does the math. Calculations come from verified constants, computed in code, and the models are told to explain those numbers, not produce their own.
- Two experts answer independently, both grounded in a curated fact base.
- Every number in both answers is checked against the fact base's ranges, and a verifier from a different model family gives a second opinion, so the third check isn't an echo of the first two.
Runs fully offline on Ollama. Zero dependencies.
Example: the tequila cell
The bundled domain is a tequila agronomy + business cell: 29 curated facts (18 numeric ranges), an agave→bottle supply-chain calculator, an agronomist and an industry analyst.
$ triple-check verify --no-llm "Agave matures in 7 years, gives 8 bottles per plant, and it takes 14 kg/l."
Claims checked: 3 (✅2 ~0 ⛔1 ?0) numeric accuracy: 0.67
CONFIRMED maturity_years = 7.0 (expected 6.0–8.0)
DISPUTED agave_kg_per_liter = 14.0 (expected 6.0–8.0) «it takes 14 kg/l.»
CONFIRMED bottles_per_plant = 8.0 (expected 7.0–10.0)
$ triple-check "How many acres would we need to match Don Julio's volume?"
CHECK 0 — verified calculation injected
CHECK 1/3 — farming
CHECK 2/3 — business
CHECK 3/3 — fact check + independent verifier
[Don Julio: 3,300,000 9L cases/yr]
VERIFIED CALCULATION (computed in Python from the fact base — not estimated):
• 3,300,000 cases/yr × 9 L = 29,700,000 L/yr = 39,600,000 bottles/yr
• Agave milled: 207,900 tonnes/yr (at 7.0 kg/L)
• Plants harvested: 4,950,000/yr (at 6.0 L/plant)
• Land harvested: 4,125 acres/yr (at 1,200 plants/acre)
• STANDING ACREAGE in production: 28,875 acres (× 7-yr maturity)
… then both experts' explanations, each number verified …
In testing, this caught a business-analyst model claiming a 9 L case = "36 liters" and "14 kg/L", flagged 6 disputed claims, and dropped the answer's confidence to 0.42 instead of passing it along.
Verdicts
| Claim verdict | Meaning |
|---|---|
CONFIRMED |
inside the fact's [lo, hi] range |
CLOSE |
outside the range but within 30% of the central value |
DISPUTED |
further off: flagged, never silently passed |
NO_ANCHOR |
nothing numeric to check against: treated as opinion |
Cell confidence is the mean numeric accuracy of both experts. Any disputed claim
multiplies it by 0.6 and rules out HIGH (≥ 0.85). MEDIUM ≥ 0.6, else LOW.
How claims are found
For every number in an answer, the checker binds it to the nearest keyword hint (within 45 characters) that points at a fact, so a number isn't matched to a keyword from another clause. Guards:
- Magnitude guard: a number is never bound to a fact more than 100× away ("8 bottles" is never checked against "3,000,000 cases")
- Sizes vs rates: "750 ml" and "5,000 acres" are sizes and are skipped. "14 kg per liter" and "7 kg/l" are rates and are checked
- Scale words: "4 million cases" → 4,000,000. Single letters ("4m") are ignored as too ambiguous
Use your own domain
from triple_check import Cell, Expert, FactBase
FACTS = {
"max_dose_mg": {"value": 4000, "unit": "mg/day", "lo": 3000, "hi": 4000,
"confidence": "HIGH", "as_of": 2024, "note": "adult acetaminophen ceiling"},
}
HINTS = [(r"per\s+day|daily|a\s+day", "max_dose_mg")]
def dose_calculator(question: str) -> str:
return "" # return a VERIFIED CALCULATION block when the question needs math
cell = Cell(
FactBase(FACTS, HINTS),
experts=[Expert("clinical", "You are a pharmacist...", "qwen2.5:3b", ("dose", "mg")),
Expert("safety", "You are a toxicologist...", "qwen2.5:3b", ("risk", "liver"))],
verifier_model="llama3.2",
calculators=[dose_calculator],
)
result = cell.triple_check("Is 6000 mg a day safe?")
print(result["confidence"], result["disputed_claims"])
See triple_check/domains/tequila.py for a complete domain.
Install
pip install triple-check-verify
ollama pull qwen2.5:3b && ollama pull llama3.2
(On PyPI the package is named triple-check-verify — triple-check was too
similar to an existing project. The command it installs is still triple-check.)
triple-check "question" ask the cell
triple-check verify [--no-llm] "text" fact-check any text
triple-check calc 4000000 [--density N] supply chain for N 9-L cases/yr
triple-check facts [term] browse the fact base
Honesty note
The tequila figures are curated estimates from public industry knowledge (CRT, Diageo/Bacardi/Becle disclosures, trade press). Each one carries a confidence level and an as-of year. They're honest ballparks, not live data. Update them in one place, the fact base, and never hard-code them in prompts.
License
MIT © Victor Navejar (Navii)
Metadata
Release files for triple-check-verify 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| triple_check_verify-0.1.1.tar.gz | 20.7 kB | With provenance | Details for triple_check_verify-0.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| triple_check_verify-0.1.1-py3-none-any.whl | Python 3 | none | any | Details for triple_check_verify-0.1.1-py3-none-any.whl |
Total release size: 40.3 kB
Release files / triple_check_verify-0.1.1.tar.gz
| Download URL | triple_check_verify-0.1.1.tar.gz |
|---|---|
| Size | 20.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
99c3b5dd61da8726cc9bb090dde3b983802700efc727cc9d42fe7d9f6f6a6988
|
|
BLAKE2b-256 checksum How to use checksums |
a175aa23e26466078e7944b4cb3bd400cbab560b6a673436e7f84873f1d9a326
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.
Transparency logRelease files / triple_check_verify-0.1.1-py3-none-any.whl
| Download URL | triple_check_verify-0.1.1-py3-none-any.whl |
|---|---|
| Size | 19.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
053e78f8bb64639e927da958bed6cb6fa215ab73ddb31caab7fa18b808df79a3
|
|
BLAKE2b-256 checksum How to use checksums |
8abacbf76f9f8738db87a8148f718ae6ea83175f49518b6c3fb9bac559a01878
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.
Transparency logSecurity for triple-check-verify 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for triple_check_verify-0.1.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.