Adversarial Honesty Tests
Adversarial honesty testing means testing an AI for honesty by trying to make it lie: every test hands the system a dishonesty shape and passes ONLY when the system fails honestly — it declines, and it discloses why. A test that confirms the happy path tells you nothing about honesty. A test that feeds a rigged input and watches the system refuse tells you everything. A silent success on a rigged input is the bug.
This is the shared method behind a family of honesty engines. It ships the vocabulary, the assertion helpers, and a catalog of the recurring shapes — each linked to a public repo that implements a probe for it.
Why the usual tests miss it
# This is decoration. It proves the tool works when nothing is wrong.
assert summarize(clean_input) == expected
# This is an honesty probe. It proves the tool refuses when something IS wrong.
result = summarize(input_with_inference_smuggled_in)
assert_honest_refusal(result) # passes ONLY if it declined AND disclosed
The vocabulary of an honest failure
from adversarial_honesty import declined, disclosed, honest_failure
honest_failure({"ok": True, "value": 42}) # False — silent success
honest_failure({"ok": False}) # False — declined but silent
honest_failure({"ok": False, "reason": "no authority"})# True — declined AND disclosed
Assert it in your tests
from adversarial_honesty import assert_honest_refusal
# require a specific disclosure, and prove nothing was fabricated on failure:
assert_honest_refusal(
scorer(insufficient_input),
must_disclose=("reason",),
must_not_fabricate=("score",), # it must NOT have invented a number
)
Run a batch
from adversarial_honesty import ProbeSuite
report = (ProbeSuite()
.probe("missing authority refuses", lambda: render(no_authority))
.probe("inference in facts refuses", lambda: render(smuggled_inference))
.run())
report["ok"], report["failed"]
A probe whose system silently passes on a rigged input FAILS the probe — you cannot accidentally write a green test here for a dishonest system.
The pattern catalog
PATTERNS names the recurring dishonesty shapes worth probing, the honest
response each demands, and a public repo that ships a probe for it:
| shape fed in | honest response | example |
|---|---|---|
| high-stakes output, no stated authority | refuse, disclose | trust-skeleton |
| inference smuggled into facts | refuse the whole render | trust-skeleton |
| narrative overwriting an observation | divert, disclose | memory-integrity |
| claim above its source's authority | demote, disclose | memory-integrity |
| a probe / check that raises | never read a crash as a pass | capability-honesty |
| "done" with no artifact | downgrade to proposal | capability-honesty |
| a scorer with insufficient input | refuse, fabricate no number | frame-sensitivity |
| a fuzzy match that could look exact | record the match kind | decision-receipts |
| a withheld decision | receipt it, content absent | decision-receipts |
checklist() renders these as a ready-to-use probe list.
Install
pip install adversarial-honesty-tests
Zero dependencies.
The Operator's Honesty Stack
This toolkit is the shared method across the stack. Each repo below ships the probes it generalizes: evidence-binding-compiler · cognitive-governance · trust-skeleton · frame-sensitivity · memory-integrity · capability-honesty · decision-receipts
License
Apache 2.0. Copyright 2026 Dave DePew Enterprises, Inc.
Metadata
Release files for adversarial-honesty-tests 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| adversarial_honesty_tests-0.1.0.tar.gz | 60.1 kB | Without provenance | Details for adversarial_honesty_tests-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| adversarial_honesty_tests-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for adversarial_honesty_tests-0.1.0-py3-none-any.whl |
Total release size: 71.5 kB
Release files / adversarial_honesty_tests-0.1.0.tar.gz
| Download URL | adversarial_honesty_tests-0.1.0.tar.gz |
|---|---|
| Size | 60.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
31f945a88293d7a4c5f157e5299ce4b3dd225fb6c38da025d22e26b054acd801
|
|
BLAKE2b-256 checksum How to use checksums |
2cc1d2bf421ceb2f3d6ad57414d7628e7248299b0e5bc4b6d7b856a7c501a9e1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.6
|
Release files / adversarial_honesty_tests-0.1.0-py3-none-any.whl
| Download URL | adversarial_honesty_tests-0.1.0-py3-none-any.whl |
|---|---|
| Size | 11.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
25eaa106c904a6ebc2b081794941936c9de870704af3b0c43f9b0ca444fbe9ce
|
|
BLAKE2b-256 checksum How to use checksums |
29692ae47bc7a1cbe4092d9c25e9af85e94c4400b07afa2d1c3054a2e3024b6b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.6
|
Security for adversarial-honesty-tests 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.