Corrobo

When an AI agent says "done", Corrobo reads the real system and rules VERIFIED, SAY-DO GAP or UNVERIFIED.

An agent's tool call can report success and still not have done the thing. Corrobo wraps the tool, reads the real before/after state through its own read-only connection, and checks it against the effect the action promised. A tool that lies, or silently fails, is caught; one that genuinely worked is proven, with a signed verdict. It never trusts the tool's own response; it reads the world.

pip install corrobo

Proof, not a promise: a live AI receptionist, verified

On 2026-09-24 Corrobo was wired to a production booking agent (Gemini + a live Postgres) and run over eight real conversations. Four callers asked for a slot that was already taken.

Verdicts matched the table in 8 of 8 conversations. It also caught its author twice on the way: a contract that said "done = any booking on this slot" was ruled VERIFIED_VACUOUS because the effect already held before the action, and a claim-reader that counted "that slot has just been booked" as a booking claim was exposed by printing its disagreements. A verifier that flags a lazy contract is the point.

Measured before that on public agent benchmarks (τ-bench retail, AgentDojo, Agent-Diff, Box): on the Box suite 59% of the agent's success claims were false, and neither the tool's return value nor a transcript-reading judge caught them. Reading the database did.

Thirty seconds, no setup

import sqlite3
from corrobo import Corrobo, SQLiteStateSource

con = sqlite3.connect("demo.db")
con.execute("CREATE TABLE IF NOT EXISTS bookings (id INTEGER PRIMARY KEY, slot TEXT UNIQUE, who TEXT)")
con.commit()

def book(slot, who):                      # a tool that LIES: says ok even when the slot was taken
    try:
        con.execute("INSERT INTO bookings (slot, who) VALUES (?, ?)", (slot, who)); con.commit()
    except sqlite3.IntegrityError:
        pass
    return {"ok": True, "message": f"You're all set for {slot}!"}

vd = Corrobo(mode="monitor", state=SQLiteStateSource(path="demo.db"))

for who in ("Alex", "Jordan"):            # second caller gets the same slot
    vd.check("book", {"slot": "Tue 3pm", "who": who}, lambda: book("Tue 3pm", who),
             succeeded=lambda r: r["ok"],
             reads=lambda a: [("bookings", {"slot": a["slot"]})],
             effect=lambda before, after, a: any(r["who"] == a["who"] for r in after["bookings"].values()))
    print(who, "->", vd.last()["status"])

# Alex -> VERIFIED
# Jordan -> SAY-DO GAP        (told "all set"; the table says otherwise)

from corrobo import Corrobo, RestStateSource

vd = Corrobo(mode="monitor", state=RestStateSource(
    "https://api.github.com",
    resources={"issue": {"path": "/repos/OWNER/REPO/issues/{number}", "pk": "number"}},
    headers={"Authorization": f"Bearer {TOKEN}", "Accept": "application/vnd.github+json"}))

vd.check("rename_issue", {"number": 1, "new_title": "Fixed"},
         lambda: github_patch(1, {"title": "Fixed"}),      # your real tool call
         succeeded=lambda r: r.ok,                           # HTTP 2xx = the tool claims success
         reads=lambda a: [("issue", {"number": a["number"]})],
         effect=lambda before, after, a: after["issue"][a["number"]]["title"] == a["new_title"])

print(vd.last().status)        # VERIFIED  |  SAY-DO GAP  |  UNVERIFIED
print(vd.last().explain())     # a human, secrets-free explanation:
#   Action:    rename_issue  (args: new_title, number)
#   Expected:  the declared effect to hold on issue[number] (read independently, before & after)
#   Observed:  real state changed [issue #1: comments, updated_at], but the promised effect did NOT hold
#   Status:    SAY-DO GAP — the real state did not match the expected effect (say-do gap)

The full stack — from "did it happen?" to "the agent can't fake success"

The say-do-gap check above is layer 1. The proven layers on top:

1. Independent verification — read real state; decide VERIFIED / SAY-DO GAP / UNVERIFIED (above).
2. Cross-system outcomes — one contract can require an effect across multiple systems of record (MultiStateSource + identity correlation). The whole invariant must hold, or it's a gap.
3. Independent verifier, out of process — run the engine as its OWN process with its OWN read-only credentials, contracts, and signing key. The agent sends only {contract_id, args} — it cannot supply the state source, the contract, or the key (VerifierService / VerifierClient).
4. Enforcement gate — an authoritative success is released only on an independent VERIFIED (OutcomeGate).
5. Signed success attestations — on a genuine VERIFIED the verifier mints a short-lived Ed25519 attestation bound to the operation; any consumer validates it with the public key (no secret). It can't be forged, modified, replayed, retargeted, or outlived (OutcomeGate.accept, verify_attestation).
6. Consumption-time re-verification — optionally re-check the effect at the moment of release to shrink the TOCTOU window (accept(..., reverify=True)).

Validated end to end on real GitHub REST, PostgreSQL, Jira Cloud, MCP (stdio), and a live receptionist agent (Gemini + Postgres), plus adversarial red-teams of the verifier trust boundary and the attestation.

Install

pip install corrobo                 # core — pure standard library, no dependencies
pip install "corrobo[postgres]"     # + PostgresStateSource   (psycopg 3)
pip install "corrobo[mcp]"          # + drive a real MCP server over stdio (mcp)
pip install "corrobo[attest]"       # + mint/verify Ed25519 success attestations (cryptography)

from corrobo import Corrobo, CorroboSayDoGap, CorroboUncertain, CorroboBlocked

Python 3.9+. The core imports nothing outside the standard library; the extras are imported lazily, so installing the core never pulls a driver you do not use.

How it works — two things you declare

Corrobo snapshots the footprint, runs the tool, snapshots again, and evaluates the effect against real state — not the tool's return value.

The verdicts

An honest tool failure is distinguished from a lie: pass succeeded=lambda r: ... (e.g. an HTTP 2xx check, or an MCP is_error flag) and a tool that reports failure becomes UNVERIFIED (action-failed), never a say-do gap.

Monitor vs enforce

• mode="monitor" (safe default) — run the tool, verify, record evidence, never interfere.
• mode="enforce" — a pre-check can block the action; a say-do gap raises CorroboSayDoGap (stop downstream); an unestablished result raises CorroboUncertain (fail-safe). An honest tool failure does not raise (its own failure result is returned).

Adapters (StateSource) — one engine, any backend

The engine reads state through a StateSource; only the adapter changes.

from corrobo import RestStateSource        # any REST/HTTP API (scoped GETs, Retry-After, retries)
from corrobo import PostgresStateSource     # real Postgres (consistent READ ONLY snapshot, PK auto-detect)
from corrobo import SQLiteStateSource       # a SQLite file or connection
from corrobo import CorroboMCPProxy         # sit in front of an existing MCP server; verify each tool call

Corrobo has been validated end-to-end against real GitHub REST, real PostgreSQL, and the real MCP SDK over stdio — the same core engine, no per-system verification logic.

Evidence

Every verification appends to a tamper-evident, hash-chained log. It stores only hashes, the argument keys, the scoped footprint (column names), and which columns changed — never raw values, never secrets, never the connection string.

vd = Corrobo(mode="monitor", state=..., evidence_path="corrobo.jsonl")
vd.evidence_log.verify()     # True — the chain is intact

Current guarantees (validated on real systems)

• VERIFIED is emitted only when the promised effect is actually present — decided by an independent read, never the tool's response body. Multi-condition effects and idempotent re-application verify correctly.
• Real say-do gaps are caught: wrong record, wrong field, partial effect, or a lying success response.
• Honest tool failures (401/403/404/HTTP errors, DB errors, tool-reported failure) → action-failed / UNVERIFIED, never a say-do gap.
• Fail-safe under unavailability: a read 404, a timeout, a dropped connection, or a resource that vanished → UNVERIFIED, never a false VERIFIED.
• Concurrency: a concurrent change to the effect-relevant state → UNVERIFIED / CorroboUncertain; unrelated concurrent churn (an extra comment, a bumped updated_at/LastModifiedDate, an unrelated row/column) does not cause a false failure.
• Direct-record reads are treated as strongly consistent; genuinely lagging reads are handled by a bounded settle window (settle_retries); rate limits honor a bounded Retry-After.

Limitations (honest)

• Concurrency detection is verdict-based, not field-traced: a concurrent write that changes a footprint row but does not flip the effect's verdict (e.g. a loose/inequality effect) is not flagged. Exact-value effects — the common case — have no blind spot.
• A read 404 is ambiguous (truly absent vs. no access), so a genuinely-absent target is reported as UNVERIFIED (fail-safe), not a definitive say-do gap.
• Corrobo is a verifier, not a lock manager or a rollback engine: it detects and (in enforce) blocks; it does not undo the tool's transaction.
• Postgres contracts must declare a reads footprint (whole-DB reads are refused for production safety), and every table read needs a primary key.
• Expiring-token refresh for REST auth is not built in yet (static headers today).

What ships in the package

corrobo/ only: the engine, the REST / Postgres / SQLite adapters, the MCP proxy, contract mining, invariants, frame conditions, drift detection, the retry and ambiguity gates, the out-of-process verifier, the enforcement gate and Ed25519 attestations. About 7,700 lines, with roughly 11,000 lines of tests behind them (83 files: engine tests plus live validation campaigns against GitHub, Postgres, Jira and MCP).

Status

Beta. The engine has had three adversarial review passes, a full CodeRabbit sweep, a public-benchmark campaign and one live production workload. The verdict logic is stable; what is still moving is deployment hardening (key distribution, a shared replay store, TLS, multi-instance) and the ergonomics people ask for. If you run an agent that writes to a real system, the most useful thing you can do is run Corrobo in monitor mode beside it for a week and tell the author what it found.

Feedback

Bugs, false verdicts, and "it should have caught this" reports are the most valuable thing you can send. Write to the author through the contact on the PyPI page, and include the explain() output and the evidence entry (both are secrets-free by construction).

License

PolyForm Noncommercial 1.0.0. Free to use, copy, modify and share for noncommercial purposes: personal projects, research, teaching, and noncommercial organizations. Commercial use needs a separate license: contact the author. See LICENSE for the full terms.