Adversarial Honesty Tests

Adversarial honesty testing means testing an AI for honesty by trying to make it lie: every test hands the system a dishonesty shape and passes ONLY when the system fails honestly — it declines, and it discloses why. A test that confirms the happy path tells you nothing about honesty. A test that feeds a rigged input and watches the system refuse tells you everything. A silent success on a rigged input is the bug.

This is the shared method behind a family of honesty engines. It ships the vocabulary, the assertion helpers, and a catalog of the recurring shapes — each linked to a public repo that implements a probe for it.

Why the usual tests miss it

# This is decoration. It proves the tool works when nothing is wrong.
assert summarize(clean_input) == expected

# This is an honesty probe. It proves the tool refuses when something IS wrong.
result = summarize(input_with_inference_smuggled_in)
assert_honest_refusal(result)     # passes ONLY if it declined AND disclosed

The vocabulary of an honest failure

from adversarial_honesty import declined, disclosed, honest_failure

honest_failure({"ok": True, "value": 42})              # False — silent success
honest_failure({"ok": False})                          # False — declined but silent
honest_failure({"ok": False, "reason": "no authority"})# True  — declined AND disclosed

Assert it in your tests

from adversarial_honesty import assert_honest_refusal

# require a specific disclosure, and prove nothing was fabricated on failure:
assert_honest_refusal(
    scorer(insufficient_input),
    must_disclose=("reason",),
    must_not_fabricate=("score",),   # it must NOT have invented a number
)

Run a batch

from adversarial_honesty import ProbeSuite

report = (ProbeSuite()
    .probe("missing authority refuses", lambda: render(no_authority))
    .probe("inference in facts refuses", lambda: render(smuggled_inference))
    .run())
report["ok"], report["failed"]

A probe whose system silently passes on a rigged input FAILS the probe — you cannot accidentally write a green test here for a dishonest system.

The pattern catalog

PATTERNS names the recurring dishonesty shapes worth probing, the honest response each demands, and a public repo that ships a probe for it:

checklist() renders these as a ready-to-use probe list.

Install

pip install adversarial-honesty-tests

Zero dependencies.

The Operator's Honesty Stack

This toolkit is the shared method across the stack. Each repo below ships the probes it generalizes: evidence-binding-compiler · cognitive-governance · trust-skeleton · frame-sensitivity · memory-integrity · capability-honesty · decision-receipts

License

Apache 2.0. Copyright 2026 Dave DePew Enterprises, Inc.