Skip to main content

triple-check

Small local LLMs are confident and bad at arithmetic. Ask a 3B model how much land a tequila brand needs and it will tell you a 9-litre case holds 36 litres.

triple-check is a pattern (and a small library) for making local-model answers trustworthy enough to act on:

  1. Python does the math. Calculations come from verified constants, computed in code, and the models are told to explain those numbers, not produce their own.
  2. Two experts answer independently, both grounded in a curated fact base.
  3. Every number in both answers is checked against the fact base's ranges, and a verifier from a different model family gives a second opinion, so the third check isn't an echo of the first two.

Runs fully offline on Ollama. Zero dependencies.

Example: the tequila cell

The bundled domain is a tequila agronomy + business cell: 29 curated facts (18 numeric ranges), an agave→bottle supply-chain calculator, an agronomist and an industry analyst.

$ triple-check verify --no-llm "Agave matures in 7 years, gives 8 bottles per plant, and it takes 14 kg/l."
Claims checked: 3  (✅2 ~0 ⛔1 ?0)   numeric accuracy: 0.67
  CONFIRMED maturity_years = 7.0 (expected 6.0–8.0)
  DISPUTED  agave_kg_per_liter = 14.0 (expected 6.0–8.0)  «it takes 14 kg/l.»
  CONFIRMED bottles_per_plant = 8.0 (expected 7.0–10.0)

$ triple-check "How many acres would we need to match Don Julio's volume?"
  CHECK 0 — verified calculation injected
  CHECK 1/3 — farming
  CHECK 2/3 — business
  CHECK 3/3 — fact check + independent verifier
  [Don Julio: 3,300,000 9L cases/yr]
  VERIFIED CALCULATION (computed in Python from the fact base — not estimated):
    • 3,300,000 cases/yr × 9 L = 29,700,000 L/yr = 39,600,000 bottles/yr
    • Agave milled: 207,900 tonnes/yr (at 7.0 kg/L)
    • Plants harvested: 4,950,000/yr (at 6.0 L/plant)
    • Land harvested: 4,125 acres/yr (at 1,200 plants/acre)
    • STANDING ACREAGE in production: 28,875 acres (× 7-yr maturity)
  … then both experts' explanations, each number verified …

In testing, this caught a business-analyst model claiming a 9 L case = "36 liters" and "14 kg/L", flagged 6 disputed claims, and dropped the answer's confidence to 0.42 instead of passing it along.

Verdicts

Claim verdict Meaning
CONFIRMED inside the fact's [lo, hi] range
CLOSE outside the range but within 30% of the central value
DISPUTED further off: flagged, never silently passed
NO_ANCHOR nothing numeric to check against: treated as opinion

Cell confidence is the mean numeric accuracy of both experts. Any disputed claim multiplies it by 0.6 and rules out HIGH (≥ 0.85). MEDIUM ≥ 0.6, else LOW.

How claims are found

For every number in an answer, the checker binds it to the nearest keyword hint (within 45 characters) that points at a fact, so a number isn't matched to a keyword from another clause. Guards:

  • Magnitude guard: a number is never bound to a fact more than 100× away ("8 bottles" is never checked against "3,000,000 cases")
  • Sizes vs rates: "750 ml" and "5,000 acres" are sizes and are skipped. "14 kg per liter" and "7 kg/l" are rates and are checked
  • Scale words: "4 million cases" → 4,000,000. Single letters ("4m") are ignored as too ambiguous

Use your own domain

from triple_check import Cell, Expert, FactBase

FACTS = {
    "max_dose_mg": {"value": 4000, "unit": "mg/day", "lo": 3000, "hi": 4000,
                    "confidence": "HIGH", "as_of": 2024, "note": "adult acetaminophen ceiling"},
}
HINTS = [(r"per\s+day|daily|a\s+day", "max_dose_mg")]

def dose_calculator(question: str) -> str:
    return ""            # return a VERIFIED CALCULATION block when the question needs math

cell = Cell(
    FactBase(FACTS, HINTS),
    experts=[Expert("clinical", "You are a pharmacist...", "qwen2.5:3b", ("dose", "mg")),
             Expert("safety", "You are a toxicologist...", "qwen2.5:3b", ("risk", "liver"))],
    verifier_model="llama3.2",
    calculators=[dose_calculator],
)
result = cell.triple_check("Is 6000 mg a day safe?")
print(result["confidence"], result["disputed_claims"])

See triple_check/domains/tequila.py for a complete domain.

Install

pip install triple-check-verify
ollama pull qwen2.5:3b && ollama pull llama3.2

(On PyPI the package is named triple-check-verify — triple-check was too similar to an existing project. The command it installs is still triple-check.)

triple-check "question"                    ask the cell
triple-check verify [--no-llm] "text"      fact-check any text
triple-check calc 4000000 [--density N]    supply chain for N 9-L cases/yr
triple-check facts [term]                  browse the fact base

Honesty note

The tequila figures are curated estimates from public industry knowledge (CRT, Diageo/Bacardi/Becle disclosures, trade press). Each one carries a confidence level and an as-of year. They're honest ballparks, not live data. Update them in one place, the fact base, and never hard-code them in prompts.

License

MIT © Victor Navejar (Navii)

Metadata

Release files for triple-check-verify 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for triple-check-verify 0.1.1
File Size Uploaded Provenance Actions
triple_check_verify-0.1.1.tar.gz 20.7 kB With provenance Details for triple_check_verify-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for triple-check-verify 0.1.1
File Interpreter ABI Platform Actions
triple_check_verify-0.1.1-py3-none-any.whl Python 3 none any Details for triple_check_verify-0.1.1-py3-none-any.whl

Total release size: 40.3 kB

Release files / triple_check_verify-0.1.1.tar.gz

Download URL triple_check_verify-0.1.1.tar.gz
Size 20.7 kB
Tags Source
SHA-256 checksum
How to use checksums
99c3b5dd61da8726cc9bb090dde3b983802700efc727cc9d42fe7d9f6f6a6988
BLAKE2b-256 checksum
How to use checksums
a175aa23e26466078e7944b4cb3bd400cbab560b6a673436e7f84873f1d9a326
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Release files / triple_check_verify-0.1.1-py3-none-any.whl

Download URL triple_check_verify-0.1.1-py3-none-any.whl
Size 19.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
053e78f8bb64639e927da958bed6cb6fa215ab73ddb31caab7fa18b808df79a3
BLAKE2b-256 checksum
How to use checksums
8abacbf76f9f8738db87a8148f718ae6ea83175f49518b6c3fb9bac559a01878
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

Security for triple-check-verify 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for triple_check_verify-0.1.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page