Capability Honesty

Capability honesty is making an AI report the lowest proven state instead of the highest hoped-for one: it advertises an ability only after a probe has exercised it and passed, and it calls work "done" only when a verifiable artifact backs the claim. An AI that says "I haven't proven I can do that yet" is worth more than one that always says yes.

Distilled from two mechanisms in a production system — a boot-conformance check that probes each ability before advertising it, and a completion gate built the day a trial agent marked a task "done" with nothing on disk. Published with its own attack suite: every test hands the system an overclaim and passes only when the ability is withheld or the completion is downgraded.

The two failures it stops

"It claims abilities it doesn't have." Registering a capability does not make it real. Here, a capability is unproven until its probe runs and passes — and a probe that crashes is never read as a pass.

from capability_honesty import CapabilityRegistry

caps = CapabilityRegistry()
caps.register("send_email", probe=lambda: smtp_connect_and_noop())

caps.require("send_email")   # ok: False — unproven, refused rather than assumed
caps.prove_all()             # run the probes
caps.require("send_email")   # ok: True  — only now, and only if the probe passed
caps.status()["advertised"]  # lists proven abilities only; the rest are 'withheld'

"It says done when it isn't." A completion claim is a claim, not a fact. settle downgrades it to a proposal unless evidence or an artifact backs it.

from capability_honesty import settle, is_done

settle("wrote the report")
# state: 'proposal' — no artifact, no verifying evidence

settle("wrote the report", evidence=lambda: Path("out/report.md").exists())
# state: 'done' — only because the check passed

settle("wrote the report", evidence=lambda: False)   # 'proposal'
settle("shipped it", artifact="out/report.md")        # 'done' — artifact present

A model can be talked into saying "done" on a bad day. The gate cannot.

The honest part

done and claimed done are different states, and this library keeps them different mechanically. A crashing probe or a raising evidence-check is treated as no proof — never optimistically as a pass. The system reports what it proved, never what it hopes.

Install

pip install capability-honesty

Zero dependencies.

Fits with

Part of The Operator's Honesty Stack — open-source honesty engines from the same production system, composed, never merged:

• trust-skeleton: a "done" that is really a proposal belongs in INFERRED, not OBSERVED.
• evidence-binding-compiler: the artifact that settles a completion is the evidence EBC would bind.
• memory-integrity: a proven completion is an observation; a claimed one is an interpretation.

License

Apache 2.0. Copyright 2026 Dave DePew Enterprises, Inc.