Skip to main content

whyfail

Evidence-based diagnostics for Python failures.

Python exceptions tell you what failed:

KeyError: 'user'

whyfail tells you why it most likely happened — by analyzing the actual failure: the traceback, the inspected runtime values, and the source around the failing line. No AI, no cloud, no network. It reads the failed program like a careful debugger would, records only what it can observe, and says "I don't have enough evidence" instead of guessing when it cannot tell.

Runtime values can contain credentials, so whyfail v2 routes every value that enters a diagnostic through SecretShield — pattern- and entropy-based secret detection — before anything is rendered. Useful crash diagnostics, without turning them into credential leaks.

KeyError: 'user'
================

Likely cause
------------
The mapping does not contain the key 'user'.

Runtime evidence
----------------
  - the subscripted value is a dict.
  - value: {'account': {'id': 7}, 'status': 'active'}
  - available keys (2): 'account', 'status'
  - the failing subscript targeted key 'user'.

Failure location
----------------
  app/users.py:4 in load()

  2 | def load():
  3 |     response = {"account": {"id": 7}, "status": "active"}
  4 |     return response["user"]
    |            ^^^^^^^^^^^^^^^^
Confidence: high

That is not generated text. Every statement in it was observed: the value's type, its keys, the requested key, and the exact source span.


Why it exists

Most error messages describe the immediate operation ("key not found"), not the reason it happened. Humans debug by looking at the runtime values around the crash — and so does whyfail, automatically, at the moment of failure while the values still exist.

whyfail is built around one rule, above all others:

It does not guess what happened. It analyzes what the failed program can actually tell us.

A mediocre tool says "your API probably changed". whyfail says:

The mapping does not contain the key 'user'.

I cannot determine why the key is missing from the available runtime evidence.

A conservative diagnosis is better than a confident but incorrect one.

Features

  • One engine, three interfaces — the CLI, the Python API, and the pytest plugin all share the same diagnostic engine.
  • Deep, but bounded, evidence — exception type/message, full chains, source context with AST analysis, safe inspection of runtime locals, function arguments, mapping keys, sequence lengths, dataclass fields and public attributes.
  • Per-exception diagnostics for KeyError, IndexError, TypeError, AttributeError, NameError/UnboundLocalError, ZeroDivisionError, ValueError, ImportError/ModuleNotFoundError, and AssertionError.
  • Exception chainsraise X from Y and implicit context are followed to the underlying failure that matters.
  • Honest source highlighting — the caret is derived from real AST + bytecode column information; when the failing expression cannot be pinned down, no misleading caret is drawn.
  • Confidence levelshigh, medium, low, and explicit insufficient evidence statements. Speculation is labelled as speculation ("Possible explanations"), never as fact.
  • SecretShield-powered redaction (v2) — every runtime value that enters a diagnostic passes through SecretShield's pattern + entropy detection before it reaches any renderer. SecretShield is installed automatically as a dependency of whyfail — you never install or configure it separately.
  • Redaction by default — local values whose names suggest secrets (password, token, api_key, authorization, private_key, cookies, ...) and values that look like credentials (sk-…, ghp_…, BEGIN ... PRIVATE KEY, JWTs, Bearer …) are never printed; high-entropy tokens with innocuous names are caught by SecretShield.
  • Fully local, deterministic, offline — no telemetry, no network, ever.

Installation

pip install whyfail

Requires Python 3.10+. Installing whyfail also installs SecretShield automatically — it is a declared dependency, so you never run pip install secretshield yourself.

whyfail 2.0

whyfail 2.0
───────────

• SecretShield-powered runtime protection
• Automatic sensitive-value redaction
• Secure nested runtime inspection
• Safer diagnostic evidence
• Existing v1 diagnostics preserved
• CLI preserved
• Python API preserved
• pytest integration preserved

(Requires Python 3.10+ — SecretShield itself requires 3.10+.)

Quick start

1. The Python API

import whyfail

try:
    run_request()
except Exception as exc:          # note: the except block is where frames live
    diagnostic = whyfail.explain(exc)
    print(whyfail.format_diagnostic(diagnostic))
    # or machine-readable:
    diagnostic.to_dict()
    diagnostic.to_json()

whyfail.explain(exception) returns a structured Diagnostic — exception chain, failure location, observed facts, likely cause, possible explanations, confidence, redaction summary. Rendering is separate, so future JSON/editor integrations need no engine changes.

2. The CLI

Run any Python program (or pytest) and get a diagnosis of the unhandled failure, after the program's own output:

whyfail run python app.py
whyfail run python -m mypackage
whyfail run pytest
$ whyfail run python app.py
Traceback (most recent call last):      # ← unchanged original output
  ...
KeyError: 'user'

====================================
whyfail diagnosis
====================================
KeyError: 'user'
...
Confidence: high

The child process runs as normally as possible: stdout, stderr, arguments, environment, and exit code are preserved. whyfail only observes; it never patches the running program.

whyfail --help
whyfail --version

3. The pytest plugin

No test rewrites needed:

pytest --whyfail

When a test fails, its diagnosis is printed alongside the normal failure output, with the failing test as context:

test_api.py::test_returns_user [call failed]
KeyError: 'user'

Failure analysis happens at the moment the exception is raised — inside pytest's own reporting hooks — so test frames and locals are still alive when the engine inspects them.

Supported diagnostics

Exception What whyfail shows (from evidence)
KeyError requested key, subject type, its keys, similar-key typo suggestions (only when a similar key actually exists)
IndexError attempted index, sequence length, valid index range
TypeError unsupported operands (from the message types + inspected operands), calling non-callables, subscripting non-subscriptables, argument-count mismatches, iterating non-iterables, None operands
AttributeError object type, requested attribute, attributes that do exist, naming-mistake suggestions backed by a similar real attribute
NameError / UnboundLocalError the missing name, scope, what is bound in that scope, bindings earlier/later in the source
ZeroDivisionError the division expression and the runtime divisor when resolvable
ValueError failed int()/float() conversions with the actual literal and runtime argument
ImportError / ModuleNotFoundError missing module vs. missing symbol, importable parent prefixes, local shadowing files, circular-import wording
AssertionError the asserted condition and the operand values that made it false; deliberate raise AssertionError is reported as such
anything else an honest generic diagnosis — facts about the location, and an explicit insufficient evidence statement

Example output

IndexError: list index out of range
===================================

Likely cause
------------
Index 10 is out of range: the sequence has 3 element(s), so valid indexes are 0..2.

Runtime evidence
----------------
  - the indexed value is a list.
  - value: ['a', 'b', 'c']
  - length: 3
  - the failing access used the index 10.

Failure location
----------------
  app/main.py:6 in main()

  4 | def main():
  5 |     items = ["a", "b", "c"]
  6 |     return items[10]
    |            ^^^^^^^^^
Confidence: high

With insufficient evidence, the output says so instead:

Likely cause
------------
Insufficient runtime evidence to determine the root cause from the available
local evidence.

Architecture

Runtime failure
       │
       ▼
Failure Capture          capture.py   — exception chains + frames (live)
       │
       ▼
Evidence Collection      runtime.py   — bounded, guarded value inspection
       │                  redact.py    — conservative secret redaction
       ▼
Context Analysis         source.py    — source reading, AST ops, carets
       │
       ▼
Diagnosis Engine         engine.py + per-exception analyzers
       │
       ▼
Diagnostic Model         models.py    — structured data, never raw values
       │
       ├── CLI renderer      renderer.py / cli.py
       ├── Python API        api.py
       └── Pytest renderer   pytest_plugin.py

The pipeline runs inside the failing process — an except block, an installed sys.excepthook (CLI child), or pytest's reporting hooks — so the exception's frames and locals are alive when inspected. Rendered diagnostics are plain text built from the structured model.

Offline capture (whyfail run)

For whyfail run python app.py, the CLI launches the program with an observing sys.excepthook. On an unhandled failure the child itself runs the engine while frames are alive, redacts, and writes the structured result to a temporary JSON sidecar that only the parent CLI reads and renders. Normal behavior — output, traceback, exit code — is untouched.

For whyfail run pytest, the CLI launches pytest with the whyfail plugin loaded; the plugin writes the same sidecar protocol and the CLI renders it. Running pytest --whyfail directly renders inline instead.

Runtime Privacy

whyfail analyzes local runtime information (traceback frames, local variables, function arguments, inspected objects) to explain failures.

Because runtime variables can contain credentials — passwords, API keys, tokens, cookies, authorization headers — whyfail integrates SecretShield to detect and redact sensitive values before diagnostic information is rendered. The sanitization boundary sits inside the engine: every string that enters the structured Diagnostic passes through SecretShield first, so the CLI, the Python API, and the pytest plugin all receive the same already-safe evidence — there is no per-renderer redaction to forget.

  • SecretShield is installed automatically as part of whyfail. It is a declared dependency; users never install or configure it separately, and security is on by default with no opt-in flag.
  • No runtime data is sent to a remote service. Detection is fully local; whyfail and SecretShield perform no network I/O.
  • Structural evidence is preserved. Redaction hides sensitive values, not the shape of the data: response is a dict, its key list, sequence lengths, and types are still reported while values are protected.
  • Fail-closed. If SecretShield is unavailable or errors, whyfail falls back to its own conservative rules (sensitive variable names and unmistakable credential shapes) and notes the fallback in the diagnostic — it never emits raw values just because the primary layer failed.
  • Redaction cannot guarantee detection of every possible secret. Both layers are heuristic; a novel secret format with an innocuous variable name may evade detection. Treat diagnostics the way you treat tracebacks.
  • Source lines are displayed like tracebacks. Context lines that contain credential literals are masked; the failing source line is shown verbatim, exactly as Python's own tracebacks show it.

Privacy and local-only guarantees

  • No network. Ever. The engine performs no I/O beyond reading local files (source) and the local import path. There is no telemetry, no external service, no analytics.
  • No raw values leave the process. Values are summarized and redacted before being recorded; the sidecar contains only rendered text.
  • whyfail never alters process behavior. Importing whyfail does not wrap or filter your program's stdout/stderr: SecretShield's stream guardians, which it installs on import, are disabled again by whyfail so the traced program's output stays byte-for-byte identical. Protection happens on the diagnostic data itself. (If you want SecretShield's stream-level protection for your own prints, import secretshield directly.)

Limitations

  • whyfail provides evidence-based likely explanations, not guaranteed root-cause analysis. The cause section states what the evidence supports, at a stated confidence; speculation appears only under "Possible explanations".
  • Analysis happens at the moment the exception is alive. If a program overrides sys.excepthook, whyfail run cannot capture that failure.
  • The interpreter must be able to import whyfail (the CLI runs children with the interpreter it is installed into).
  • A pathological object whose __repr__ loops forever without raising cannot be interrupted from inside the same process (standard for any diagnostic tool); all raising/misbehaving cases are fully guarded.
  • Source files are read from disk at analysis time, so source edits made between a crash and the analysis can make code context stale.
  • Secret redaction is heuristic and not perfect (see Runtime Privacy). whyfail hides sensitive values that SecretShield's patterns and entropy detection — plus whyfail's own name rules — recognize; it cannot guarantee that an unrecognized secret format never appears.
  • Python 3.10+ is required (whyfail 2 depends on SecretShield, which requires 3.10+).

Development

git clone https://github.com/Sam3360/whyfail
cd whyfail
pip install -e ".[dev]"
python -m pytest

Project layout

pyproject.toml           packaging / metadata / entry points
src/whyfail/             the package (engine, analyzers, renderers, CLI, plugin)
tests/                   unit + integration + false-positive tests
.github/workflows/ci.yml CI (Linux, Python 3.10–3.14, build check)

Testing

The suite covers every supported exception type, runtime inspection, source extraction and highlighting, exception chaining, confidence, redaction, hostile/recursive/huge values, the CLI (exit codes, output preservation, -m and pytest invocations), the pytest plugin, and — importantly — false-positive tests proving the engine refuses to invent causes when the evidence does not support them.

Releasing

python -m build
twine upload dist/*     # after review

License

MIT — see LICENSE.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

whyfail-2.0.0.tar.gz (78.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

whyfail-2.0.0-py3-none-any.whl (70.0 kB view details)

Uploaded Python 3

File details

Details for the file whyfail-2.0.0.tar.gz.

File metadata

  • Download URL: whyfail-2.0.0.tar.gz
  • Upload date:
  • Size: 78.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.14.2

File hashes

Hashes for whyfail-2.0.0.tar.gz
Algorithm Hash digest
SHA256 ecf78756d06af3b4772fb48ae5d4dd337f2fb879f9bc0f0cb896ecf9b203d670
MD5 007f835587367e834de97e7bcb29a082
BLAKE2b-256 00d534e7a4067e9ab44f6eaf27fef0769e43fd2563f80870764fff6418f0b043

See more details on using hashes here.

File details

Details for the file whyfail-2.0.0-py3-none-any.whl.

File metadata

  • Download URL: whyfail-2.0.0-py3-none-any.whl
  • Upload date:
  • Size: 70.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.14.2

File hashes

Hashes for whyfail-2.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f77d04e3d08838f275154810be4fcbe6a1b13f1f59eda941250b800e5529a34e
MD5 b194dad36e5f59eefb12583ca0ab6c97
BLAKE2b-256 4afb9744903a0eb2c3072662fbab709b07dba796adf105632abcfddf2c444198

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

2.0.0 This release

2 files

1.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page