credence-guard 1.2.5

Credence — epistemic enforcement layer that prevents LLMs from forgetting what they didn't know

Credence

AI doesn't remember what it wasn't sure about. Credence does.

pip install credence-guard
credence demo   # 30-second smoke test, no API key required

[mcp] adds the FastMCP server for Claude Code. Core package has zero hard dependencies.

---

The problem

You say: "The rate limit is probably around 50 — I haven't confirmed it yet."

Fifteen turns later, Claude writes:

RATE_LIMIT = 50   # no warning. no flag. shipped.

The API rejects every request at 2am. The real limit was 10. Claude forgot you weren't sure.

This isn't hallucination. The model reproduced exactly what it read. What it read had the qualifier stripped — by context compression, fifteen turns back.

---

What Credence does

Tracks uncertain values the moment you state them. Blocks writes that embed those values until you confirm them.

you say "rate limit is probably 50"
    → observer registers it (before Claude responds)
    → Claude writes: RATE_LIMIT = 50  # ⚠ CREDENCE[unverified]
    → write blocked until you confirm

Every other tool warns. Credence enforces.

---

What it looks like

# Claude generates this. Credence intercepts before it ships.

class StripeClient:
    API_VERSION  = "2023-10-16"  # ⚠⚠ CREDENCE[stale]: API date versions change on release — verify before shipping
    RATE_LIMIT   = 100           # ⚠  CREDENCE[unverified]: I think Stripe rate limit is around 100 req/min
    TOKEN_EXPIRY = 3600          # ⚠⚠ CREDENCE[stale]: Token/session lifetime values are set by the vendor — verify
    MAX_RETRIES  = 3
    TIMEOUT_MS   = 5000

credence: blocked Edit — 2 unverified value(s)
  → I think Stripe rate limit is around 100 req/min | TOKEN_EXPIRY = 3600
  Verify first, then retry. Use credence_constraints to see all pending.

After you confirm: "Confirmed — rate limit is 100 req/min per stripe.com/docs" → gate clears.

---

Setup

1. Add to .mcp.json:

{ "mcpServers": { "credence": { "command": "credence-server" } } }

2. Add to .claude/settings.json:

{
  "hooks": {
    "UserPromptSubmit": [
      { "hooks": [{ "type": "command", "command": "python3 -m credence.observer" }] }
    ],
    "PreToolUse": [
      {
        "matcher": "Write|Edit|Bash|NotebookEdit",
        "hooks": [{ "type": "command", "command": "python3 -m credence.hooks" }]
      }
    ]
  }
}

Done. No API key required.

Registry: Credence creates epistemic_registry.db in your working directory. Add *.db to your .gitignore, or set CREDENCE_DB=~/.credence/registry.db to keep it global.

Session tracking: Set CREDENCE_SESSION_ID=my-project to keep constraints stable across directory changes and terminal restarts.

Event log: The gate writes block/allow events to ~/.credence/events.jsonl (local only, never sent anywhere). Set CREDENCE_NO_LOG=1 to disable.

Constraint cap: The registry allows up to 500 constraints per session by default. Override with CREDENCE_MAX_CONSTRAINTS=<n>.

---

How it works

Two layers, neither requires model cooperation:

Layer	Hook	Role
Observer	UserPromptSubmit	Passive listener — registers uncertain values before Claude generates anything
Gate	PreToolUse	Blocks writes that embed unverified values

The observer fires before the model processes your message. If you say "I think the rate limit is 50", the registry has that entry before Claude generates a single token.

---

What gets blocked

credence: blocked Edit — 2 unverified value(s)
  → rate limit is probably 50 req/min | token expires in 3600s
  Verify first, then retry. Use credence_constraints to see all pending.

Once verified, the gate clears.

---

What Credence does NOT do

• Does not verify facts — it cannot tell you if a value is correct
• Does not catch uncertainty that was never stated
• Does not block the model from saying a wrong value in prose — only from writing it to a file or command

---

Measured results

46% of uncertainty qualifiers are stripped by Claude Haiku during context compression. Credence blocks 100% of those writes (n=50, bootstrap CI: [0%–0%]).

Validated across 7 open-weight models (Qwen, Mistral, Llama, Phi, Gemma) from 5 organizations: same failure mode, same block rate.

credence demo                     # smoke test, no API key
credence stats                    # false-positive rate from real gate usage
credence feedback 1|2|3           # tag last gate block: correct / noise / skip
python3 -m pytest tests/ -q       # 829 tests
python3 -m evals.latency_report   # P50/P95/P99

Full methodology: docs/TECHNICAL_REPORT.md

---

Project layout

credence/         pip-installable package
  observer.py     passive UserPromptSubmit hook
  hooks.py        PreToolUse enforcement gate
  mcp_server.py   17-tool MCP server
  registry.py     SQLite constraint store
  memory.py       cross-session persistence
tests/            829 tests
evals/            validation studies + multi-model benchmarks
docs/             technical report, architecture, ETP spec
credence_gate/    Rust gate (alternative to Python hooks.py)
experimental/     Phase 2 work — not yet shipped
paper/            Research paper draft + figures

---

Built by

Lakshmi Chakradhar Vijayarao — GitHub · LinkedIn · X

MIT License