whyfail
Evidence-based diagnostics for Python failures.
Python exceptions tell you what failed:
KeyError: 'user'
whyfail tells you why it most likely happened — by analyzing the actual
failure: the traceback, the inspected runtime values, and the source around
the failing line. No AI, no cloud, no network. It reads the failed program
like a careful debugger would, records only what it can observe, and says
"I don't have enough evidence" instead of guessing when it cannot tell.
Runtime values can contain credentials, so whyfail v2 routes every value that enters a diagnostic through SecretShield — pattern- and entropy-based secret detection — before anything is rendered. Useful crash diagnostics, without turning them into credential leaks.
KeyError: 'user'
================
Likely cause
------------
The mapping does not contain the key 'user'.
Runtime evidence
----------------
- the subscripted value is a dict.
- value: {'account': {'id': 7}, 'status': 'active'}
- available keys (2): 'account', 'status'
- the failing subscript targeted key 'user'.
Failure location
----------------
app/users.py:4 in load()
2 | def load():
3 | response = {"account": {"id": 7}, "status": "active"}
4 | return response["user"]
| ^^^^^^^^^^^^^^^^
Confidence: high
That is not generated text. Every statement in it was observed: the value's type, its keys, the requested key, and the exact source span.
Why it exists
Most error messages describe the immediate operation ("key not found"), not
the reason it happened. Humans debug by looking at the runtime values around
the crash — and so does whyfail, automatically, at the moment of failure
while the values still exist.
whyfail is built around one rule, above all others:
It does not guess what happened. It analyzes what the failed program can actually tell us.
A mediocre tool says "your API probably changed". whyfail says:
The mapping does not contain the key 'user'.
I cannot determine why the key is missing from the available runtime evidence.
A conservative diagnosis is better than a confident but incorrect one.
Features
- One engine, three interfaces — the CLI, the Python API, and the pytest plugin all share the same diagnostic engine.
- Deep, but bounded, evidence — exception type/message, full chains, source context with AST analysis, safe inspection of runtime locals, function arguments, mapping keys, sequence lengths, dataclass fields and public attributes.
- Per-exception diagnostics for
KeyError,IndexError,TypeError,AttributeError,NameError/UnboundLocalError,ZeroDivisionError,ValueError,ImportError/ModuleNotFoundError, andAssertionError. - Exception chains —
raise X from Yand implicit context are followed to the underlying failure that matters. - Honest source highlighting — the caret is derived from real AST + bytecode column information; when the failing expression cannot be pinned down, no misleading caret is drawn.
- Confidence levels —
high,medium,low, and explicit insufficient evidence statements. Speculation is labelled as speculation ("Possible explanations"), never as fact. - SecretShield-powered redaction (v2) — every runtime value that enters a diagnostic passes through SecretShield's pattern + entropy detection before it reaches any renderer. SecretShield is installed automatically as a dependency of whyfail — you never install or configure it separately.
- Redaction by default — local values whose names suggest secrets
(password, token, api_key, authorization, private_key, cookies, ...) and
values that look like credentials (
sk-…,ghp_…,BEGIN ... PRIVATE KEY, JWTs,Bearer …) are never printed; high-entropy tokens with innocuous names are caught by SecretShield. - Fully local, deterministic, offline — no telemetry, no network, ever.
Installation
pip install whyfail
Requires Python 3.10+. Installing whyfail also installs SecretShield
automatically — it is a declared dependency, so you never run
pip install secretshield yourself.
whyfail 2.0
whyfail 2.0
───────────
• SecretShield-powered runtime protection
• Automatic sensitive-value redaction
• Secure nested runtime inspection
• Safer diagnostic evidence
• Existing v1 diagnostics preserved
• CLI preserved
• Python API preserved
• pytest integration preserved
(Requires Python 3.10+ — SecretShield itself requires 3.10+.)
Quick start
1. The Python API
import whyfail
try:
run_request()
except Exception as exc: # note: the except block is where frames live
diagnostic = whyfail.explain(exc)
print(whyfail.format_diagnostic(diagnostic))
# or machine-readable:
diagnostic.to_dict()
diagnostic.to_json()
whyfail.explain(exception) returns a structured
Diagnostic — exception chain, failure location,
observed facts, likely cause, possible explanations, confidence, redaction
summary. Rendering is separate, so future JSON/editor integrations need no
engine changes.
2. The CLI
Run any Python program (or pytest) and get a diagnosis of the unhandled failure, after the program's own output:
whyfail run python app.py
whyfail run python -m mypackage
whyfail run pytest
$ whyfail run python app.py
Traceback (most recent call last): # ← unchanged original output
...
KeyError: 'user'
====================================
whyfail diagnosis
====================================
KeyError: 'user'
...
Confidence: high
The child process runs as normally as possible: stdout, stderr, arguments,
environment, and exit code are preserved. whyfail only observes; it never
patches the running program.
whyfail --help
whyfail --version
3. The pytest plugin
No test rewrites needed:
pytest --whyfail
When a test fails, its diagnosis is printed alongside the normal failure output, with the failing test as context:
test_api.py::test_returns_user [call failed]
KeyError: 'user'
Failure analysis happens at the moment the exception is raised — inside pytest's own reporting hooks — so test frames and locals are still alive when the engine inspects them.
Supported diagnostics
| Exception | What whyfail shows (from evidence) |
|---|---|
KeyError |
requested key, subject type, its keys, similar-key typo suggestions (only when a similar key actually exists) |
IndexError |
attempted index, sequence length, valid index range |
TypeError |
unsupported operands (from the message types + inspected operands), calling non-callables, subscripting non-subscriptables, argument-count mismatches, iterating non-iterables, None operands |
AttributeError |
object type, requested attribute, attributes that do exist, naming-mistake suggestions backed by a similar real attribute |
NameError / UnboundLocalError |
the missing name, scope, what is bound in that scope, bindings earlier/later in the source |
ZeroDivisionError |
the division expression and the runtime divisor when resolvable |
ValueError |
failed int()/float() conversions with the actual literal and runtime argument |
ImportError / ModuleNotFoundError |
missing module vs. missing symbol, importable parent prefixes, local shadowing files, circular-import wording |
AssertionError |
the asserted condition and the operand values that made it false; deliberate raise AssertionError is reported as such |
| anything else | an honest generic diagnosis — facts about the location, and an explicit insufficient evidence statement |
Example output
IndexError: list index out of range
===================================
Likely cause
------------
Index 10 is out of range: the sequence has 3 element(s), so valid indexes are 0..2.
Runtime evidence
----------------
- the indexed value is a list.
- value: ['a', 'b', 'c']
- length: 3
- the failing access used the index 10.
Failure location
----------------
app/main.py:6 in main()
4 | def main():
5 | items = ["a", "b", "c"]
6 | return items[10]
| ^^^^^^^^^
Confidence: high
With insufficient evidence, the output says so instead:
Likely cause
------------
Insufficient runtime evidence to determine the root cause from the available
local evidence.
Architecture
Runtime failure
│
▼
Failure Capture capture.py — exception chains + frames (live)
│
▼
Evidence Collection runtime.py — bounded, guarded value inspection
│ redact.py — conservative secret redaction
▼
Context Analysis source.py — source reading, AST ops, carets
│
▼
Diagnosis Engine engine.py + per-exception analyzers
│
▼
Diagnostic Model models.py — structured data, never raw values
│
├── CLI renderer renderer.py / cli.py
├── Python API api.py
└── Pytest renderer pytest_plugin.py
The pipeline runs inside the failing process — an except block, an
installed sys.excepthook (CLI child), or pytest's reporting hooks — so the
exception's frames and locals are alive when inspected. Rendered diagnostics
are plain text built from the structured model.
Offline capture (whyfail run)
For whyfail run python app.py, the CLI launches the program with an
observing sys.excepthook. On an unhandled failure the child itself runs
the engine while frames are alive, redacts, and writes the structured result
to a temporary JSON sidecar that only the parent CLI reads and renders.
Normal behavior — output, traceback, exit code — is untouched.
For whyfail run pytest, the CLI launches pytest with the whyfail plugin
loaded; the plugin writes the same sidecar protocol and the CLI renders it.
Running pytest --whyfail directly renders inline instead.
Runtime Privacy
whyfail analyzes local runtime information (traceback frames, local variables, function arguments, inspected objects) to explain failures.
Because runtime variables can contain credentials — passwords, API keys,
tokens, cookies, authorization headers — whyfail integrates SecretShield
to detect and redact sensitive values before diagnostic information is
rendered. The sanitization boundary sits inside the engine: every string that
enters the structured Diagnostic passes through SecretShield first, so the
CLI, the Python API, and the pytest plugin all receive the same already-safe
evidence — there is no per-renderer redaction to forget.
- SecretShield is installed automatically as part of whyfail. It is a declared dependency; users never install or configure it separately, and security is on by default with no opt-in flag.
- No runtime data is sent to a remote service. Detection is fully local; whyfail and SecretShield perform no network I/O.
- Structural evidence is preserved. Redaction hides sensitive values,
not the shape of the data:
response is a dict, its key list, sequence lengths, and types are still reported while values are protected. - Fail-closed. If SecretShield is unavailable or errors, whyfail falls back to its own conservative rules (sensitive variable names and unmistakable credential shapes) and notes the fallback in the diagnostic — it never emits raw values just because the primary layer failed.
- Redaction cannot guarantee detection of every possible secret. Both layers are heuristic; a novel secret format with an innocuous variable name may evade detection. Treat diagnostics the way you treat tracebacks.
- Source lines are displayed like tracebacks. Context lines that contain credential literals are masked; the failing source line is shown verbatim, exactly as Python's own tracebacks show it.
Privacy and local-only guarantees
- No network. Ever. The engine performs no I/O beyond reading local files (source) and the local import path. There is no telemetry, no external service, no analytics.
- No raw values leave the process. Values are summarized and redacted before being recorded; the sidecar contains only rendered text.
- whyfail never alters process behavior. Importing whyfail does not wrap
or filter your program's stdout/stderr: SecretShield's stream guardians,
which it installs on import, are disabled again by whyfail so the traced
program's output stays byte-for-byte identical. Protection happens on the
diagnostic data itself. (If you want SecretShield's stream-level
protection for your own prints,
import secretshielddirectly.)
Limitations
whyfailprovides evidence-based likely explanations, not guaranteed root-cause analysis. The cause section states what the evidence supports, at a stated confidence; speculation appears only under "Possible explanations".- Analysis happens at the moment the exception is alive. If a program
overrides
sys.excepthook,whyfail runcannot capture that failure. - The interpreter must be able to import
whyfail(the CLI runs children with the interpreter it is installed into). - A pathological object whose
__repr__loops forever without raising cannot be interrupted from inside the same process (standard for any diagnostic tool); all raising/misbehaving cases are fully guarded. - Source files are read from disk at analysis time, so source edits made between a crash and the analysis can make code context stale.
- Secret redaction is heuristic and not perfect (see Runtime Privacy). whyfail hides sensitive values that SecretShield's patterns and entropy detection — plus whyfail's own name rules — recognize; it cannot guarantee that an unrecognized secret format never appears.
- Python 3.10+ is required (whyfail 2 depends on SecretShield, which requires 3.10+).
Development
git clone https://github.com/Sam3360/whyfail
cd whyfail
pip install -e ".[dev]"
python -m pytest
Project layout
pyproject.toml packaging / metadata / entry points
src/whyfail/ the package (engine, analyzers, renderers, CLI, plugin)
tests/ unit + integration + false-positive tests
.github/workflows/ci.yml CI (Linux, Python 3.10–3.14, build check)
Testing
The suite covers every supported exception type, runtime inspection, source
extraction and highlighting, exception chaining, confidence, redaction,
hostile/recursive/huge values, the CLI (exit codes, output preservation,
-m and pytest invocations), the pytest plugin, and — importantly —
false-positive tests proving the engine refuses to invent causes when the
evidence does not support them.
Releasing
python -m build
twine upload dist/* # after review
License
MIT — see LICENSE.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file whyfail-2.0.0.tar.gz.
File metadata
- Download URL: whyfail-2.0.0.tar.gz
- Upload date:
- Size: 78.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.14.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ecf78756d06af3b4772fb48ae5d4dd337f2fb879f9bc0f0cb896ecf9b203d670
|
|
| MD5 |
007f835587367e834de97e7bcb29a082
|
|
| BLAKE2b-256 |
00d534e7a4067e9ab44f6eaf27fef0769e43fd2563f80870764fff6418f0b043
|
File details
Details for the file whyfail-2.0.0-py3-none-any.whl.
File metadata
- Download URL: whyfail-2.0.0-py3-none-any.whl
- Upload date:
- Size: 70.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.14.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
f77d04e3d08838f275154810be4fcbe6a1b13f1f59eda941250b800e5529a34e
|
|
| MD5 |
b194dad36e5f59eefb12583ca0ab6c97
|
|
| BLAKE2b-256 |
4afb9744903a0eb2c3072662fbab709b07dba796adf105632abcfddf2c444198
|