CertainLogic Verifier – Open‑source deterministic AI verification

🚀 Try in 2 Minutes • 🎯 Why • 🏗️ Architecture • 📈 Benchmarks • 📊 Comparison • ⚡ Quick Start • 🐳 Deployment • 📖 API • 🛡️ Compliance • 📅 Roadmap

🚀 Try in 2 Minutes

Copy‑paste this in your terminal:

git clone https://github.com/CertainLogicAI/hallucination-guard.git
cd hallucination-guard
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 8000

In another terminal, test validation:

curl -X POST http://localhost:8000/validate \\
  -H "Content-Type: application/json" \\
  -d '{"query": "What is the price of GPT‑5?", "response": "$200/month"}'

{
  "valid": false,
  "confidence": 0.5,
  "severity": "medium",
  "message": "Factual mismatch: No matching fact for factual query — unverifiable",
  "flags": ["Specific claim with no verifiable fact — flagged for human review"]
}

🎯 Why This Exists

AI hallucinations break trust and compliance. But most “guardrail” tools are black‑box SaaS that create new risks: no auditability, data‑residency concerns, and vendor lock‑in.

CertainLogic Verifier is different:

• ✅ Deterministic verification – rule‑based fact‑checking against your versioned facts DB (no extra LLM calls)
• ✅ Up to 98 % token reduction – semantic caching + similarity lookup bypass LLMs entirely
• ✅ Self‑hosted & air‑gapped – runs entirely inside your VPC, on‑prem, or private cloud
• ✅ Regulatory‑ready – built‑in audit logging, SBOM, and deployment patterns for HIPAA/GDPR/SOC2/FedRAMP
• ✅ MIT licensed – every line inspectable by your security/compliance teams

Built for regulated industries (healthcare, finance, government) and cost‑conscious AI agent teams that need trustworthy AI without sacrificing control.

📈 Benchmarks (Real‑World Performance)

Based on 62‑example benchmark suite (April 2026). New qualifier safelist and unit‑aware matching push accuracy >85 %.

📊 Comparison: Deterministic vs. Probabilistic Guardrails

Bottom line: We give you a verifiable safety layer that doesn’t hallucinate and doesn’t add cost.

🏗️ Architecture

Query → [Intent Router] → [Semantic Cache] → Cache Hit → Bypass LLM (0 tokens)
                ↓ (miss)
           [Token Reduction] → [Hallucination Detector] → [Facts DB]
                ↓
           LLM → Response → [Audit Log (SHA‑256 chained)]

Components included:

• Hallucination Detector – factual consistency, uncertainty detection, internal contradiction checks
• Token Reduction Engine – SQLite LRU cache + semantic similarity + summarization fallback
• Semantic Cache (L2) – sentence‑transformers embeddings for similarity lookup
• Deterministic Memory Search – TF‑IDF over local .md files (no embeddings needed)
• Intent Classifier/Router – zero‑LLM rule‑based routing to appropriate models
• FastAPI Service – production‑ready REST API with metrics, audit logging, health checks

⚡ Quick Start

1. Clone & Install

git clone https://github.com/CertainLogicAI/hallucination-guard.git
cd hallucination-guard
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -r requirements.txt

2. Run the Service

export FACTS_DB_PATH=./facts_db.json
uvicorn main:app --host 0.0.0.0 --port 8000

3. Validate Your First Query

curl -X POST http://localhost:8000/validate \\
  -H "Content-Type: application/json" \\
  -d '{"query": "What is 2+2?", "response": "The answer is 5."}'

4. Reduce Token Count (Save Money)

curl -X POST http://localhost:8000/reduce \\
  -H "Content-Type: application/json" \\
  -d '{"query": "Explain quantum entanglement in simple terms...", "semantic": true}'

🐳 Deployment

Docker (Single Container)

FROM python:3.11-slim
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Kubernetes (Helm)

Example Helm chart included in deploy/helm/ (coming soon).

Air‑Gapped / On‑Premises

1. Build Docker image inside your secure network
2. Push to private registry
3. Deploy with persistent volume for cache.db and facts_db.json
4. Configure network policies to block all egress (no external API calls)

📖 API Reference

POST /validate

Validate an AI-generated response against the facts database.

curl -X POST http://localhost:8000/validate \
  -H "Content-Type: application/json" \
  -d '{"query": "What is 2+2?", "response": "4"}'

Request body:

Response:

{
  "query": "What is 2+2?",
  "valid": true,
  "flagged": false,
  "confidence": 1.0,
  "severity": "none",
  "flags": [],
  "checks": {
    "factual_consistency": {"passed": true, "message": "...", "score": 1.0},
    "uncertainty": {"passed": true, "issues": [], "score": 1.0},
    "internal_consistency": {"passed": true, "issues": [], "score": 1.0},
    "specificity": {"passed": true, "message": "...", "score": 1.0}
  }
}

POST /reduce

Reduce token count via caching and deterministic summarization.

curl -X POST http://localhost:8000/reduce \
  -H "Content-Type: application/json" \
  -d '{"query": "Explain quantum theory in detail", "semantic": true}'

POST /search

Search verified facts via TF-IDF over the memory index.

curl -X POST http://localhost:8000/search \
  -H "Content-Type: application/json" \
  -d '{"query": "Python best practices", "top_k": 5}'

POST /route

Classify a query and route to the appropriate handler.

curl -X POST http://localhost:8000/route \
  -H "Content-Type: application/json" \
  -d '{"query": "What is the price of GPT-5?"}'

Response includes: brain_handler, openclaw_model, compressed query, token_count, full intent classification.

GET /health

Health check. Returns {"status": "ok"} when the service is running.

GET /metrics

Cache hit rates, token savings, cost tracking, and query volumes.

DELETE /cache

Purge the token-reduction cache. Returns {"cleared": true}.

🔧 Extending the Facts Database

The facts database is a versioned JSON file:

{
  "facts": {
    "python release year": {
      "type": "numeric",
      "value": "1991"
    },
    "speed of light": {
      "type": "numeric",
      "value": "299792458",
      "unit": "m/s"
    },
    "capital of france": {
      "type": "string",
      "value": "paris"
    },
    "product price": {
      "type": "numeric",
      "value": "49.99",
      "unit": "usd",
      "tolerance": 0.01
    }
  }
}

Fact schema:

Workflow:

1. Export internal knowledge (prices, policies, compliance rules) to JSON
2. Load via FACTS_DB_PATH environment variable or pass to HallucinationDetector(facts_db_path=...)
3. The detector flags any AI response contradicting these facts
4. See examples/ for working code samples

🔌 Integration Examples

LangChain (built-in)

pip install hallucination-guard langchain-core

Pattern 1 — Callback handler (drop-in, validates every LLM response):

from langchain_openai import ChatOpenAI
from hallucination_guard.integrations.langchain import HallucinationGuardCallback

callback = HallucinationGuardCallback(
    facts_db_path="./company_facts.json",
    raise_on_hallucination=True,  # block hallucinated responses
)

llm = ChatOpenAI(callbacks=[callback])
llm.invoke("What is our enterprise pricing?")  # validated automatically

Pattern 2 — LCEL Runnable (compose into pipelines):

from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from hallucination_guard.integrations.langchain import HallucinationGuardChain

guard = HallucinationGuardChain(facts_db_path="./facts.json")

chain = ChatOpenAI() | StrOutputParser() | guard.as_runnable()
result = chain.invoke("What is 2+2?")  # hallucinations blocked

See examples/langchain_integration.py for a complete working demo.

Direct Python

from hallucination_guard import HallucinationDetector

detector = HallucinationDetector(facts_db_path="./company_facts.json")
result = detector.validate("What is 2+2?", "4")
assert result["valid"] is True

FastAPI Middleware

from fastapi import FastAPI, Request
app = FastAPI()

@app.middleware("http")
async def verify_ai_output(request: Request, call_next):
    response = await call_next(request)
    # Extract query/response, validate, log/block invalid outputs
    return response

Airflow / Prefect

from token_reduction_engine import reduce_tokens

def compress_query(task_instance):
    query = task_instance.xcom_pull(task_ids="previous")
    reduced = reduce_tokens(query, semantic=True)
    task_instance.xcom_push(key="compressed_query", value=reduced["reduced_query"])

🛡️ Compliance & Security

Audit Trail

Every validation logged to append‑only JSONL with SHA‑256 hash chaining (see examples/audit_logger.py).

Data Residency

Zero data exfiltration – runs entirely inside your VPC, private cloud, or air‑gapped network.

SBOM & Vulnerability Scanning

Software Bill of Materials in sbom.spdx.json, regularly updated with vulnerability reports.

Certification Support

Designed for:

• HIPAA – No PHI exfiltration, audit logging, access controls
• GDPR – Data locality, right to erasure (cache clearing), transparency
• SOC2 – Security, availability, processing integrity
• FedRAMP – Controlled environments, no external dependencies

📅 Roadmap

• Q2 2026 – GPU‑accelerated embedding backfill, PostgreSQL vector store support
• Q3 2026 – Multi‑modal verification (image, audio, video), real‑time streaming validation
• Q4 2026 – Federated learning for fact‑database sharing (enterprise‑only)

💼 Coder Pack — Production-Ready in Minutes

The free tier includes 100 verified facts and 10 sample queries — enough to prove the system works and see exact token savings.

Want to skip weeks of DIY cache warming and fact verification?

What's in the pack:

• 303+ verified facts across Python, JS/TS, Docker, Git, SQL, HTTP, Cloud, Security, DevOps, React, FastAPI
• Pre-warmed semantic cache from thousands of verified queries
• Drop-in cache.db replacement — zero cold start
• Every fact sourced and dated

💡 $69 is less than most developers spend on a single day of LLM API calls during cache warming.

# Example: query that hits the facts cache (0 tokens, $0.00)
curl -X POST http://localhost:8000/reduce \
  -H "Content-Type: application/json" \
  -d '{"query": "What is the current stable version of Python?", "semantic": true}'

Coming soon: Industry packs for Healthcare (HIPAA/FDA), Finance (SOX/PCI), and Industrial Automation (IEC/ISO).

We also provide enterprise cache‑warming services — we ingest your internal docs and deliver a production‑ready verified cache ($999–$5,000+/project).

Contact: sales@certainlogic.ai | @CertainLogicAI

📄 License

MIT License – see LICENSE for details.

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Built with transparency, for trust.