whyfail

Evidence-based diagnostics for Python failures.

Python exceptions tell you what failed:

KeyError: 'user'

whyfail tells you why it most likely happened — by analyzing the actual failure: the traceback, the inspected runtime values, and the source around the failing line. No AI, no cloud, no network. It reads the failed program like a careful debugger would, records only what it can observe, and says "I don't have enough evidence" instead of guessing when it cannot tell.

KeyError: 'user'
================

Likely cause
------------
The mapping does not contain the key 'user'.

Runtime evidence
----------------
  - the subscripted value is a dict.
  - value: {'account': {'id': 7}, 'status': 'active'}
  - available keys (2): 'account', 'status'
  - the failing subscript targeted key 'user'.

Failure location
----------------
  app/users.py:4 in load()

  2 | def load():
  3 |     response = {"account": {"id": 7}, "status": "active"}
  4 |     return response["user"]
    |            ^^^^^^^^^^^^^^^^
Confidence: high

That is not generated text. Every statement in it was observed: the value's type, its keys, the requested key, and the exact source span.

---

Why it exists

Most error messages describe the immediate operation ("key not found"), not the reason it happened. Humans debug by looking at the runtime values around the crash — and so does whyfail, automatically, at the moment of failure while the values still exist.

whyfail is built around one rule, above all others:

It does not guess what happened. It analyzes what the failed program can actually tell us.

A mediocre tool says "your API probably changed". whyfail says:

The mapping does not contain the key 'user'.

I cannot determine why the key is missing from the available runtime evidence.

A conservative diagnosis is better than a confident but incorrect one.

Features

• One engine, three interfaces — the CLI, the Python API, and the pytest plugin all share the same diagnostic engine.
• Deep, but bounded, evidence — exception type/message, full chains, source context with AST analysis, safe inspection of runtime locals, function arguments, mapping keys, sequence lengths, dataclass fields and public attributes.
• Per-exception diagnostics for KeyError, IndexError, TypeError, AttributeError, NameError/UnboundLocalError, ZeroDivisionError, ValueError, ImportError/ModuleNotFoundError, and AssertionError.
• Exception chains — raise X from Y and implicit context are followed to the underlying failure that matters.
• Honest source highlighting — the caret is derived from real AST + bytecode column information; when the failing expression cannot be pinned down, no misleading caret is drawn.
• Confidence levels — high, medium, low, and explicit insufficient evidence statements. Speculation is labelled as speculation ("Possible explanations"), never as fact.
• Redaction by default — local values whose names suggest secrets (password, token, api_key, authorization, private_key, cookies, ...) and values that look like credentials (sk-…, ghp_…, BEGIN ... PRIVATE KEY, JWTs, Bearer …) are never printed.
• Fully local, deterministic, offline — zero runtime dependencies (Python standard library only), no telemetry, no network, ever.

---

Installation

pip install whyfail

Requires Python 3.9+.

Quick start

1. The Python API

import whyfail

try:
    run_request()
except Exception as exc:          # note: the except block is where frames live
    diagnostic = whyfail.explain(exc)
    print(whyfail.format_diagnostic(diagnostic))
    # or machine-readable:
    diagnostic.to_dict()
    diagnostic.to_json()

whyfail.explain(exception) returns a structured Diagnostic — exception chain, failure location, observed facts, likely cause, possible explanations, confidence, redaction summary. Rendering is separate, so future JSON/editor integrations need no engine changes.

2. The CLI

Run any Python program (or pytest) and get a diagnosis of the unhandled failure, after the program's own output:

whyfail run python app.py
whyfail run python -m mypackage
whyfail run pytest

$ whyfail run python app.py
Traceback (most recent call last):      # ← unchanged original output
  ...
KeyError: 'user'

====================================
whyfail diagnosis
====================================
KeyError: 'user'
...
Confidence: high

The child process runs as normally as possible: stdout, stderr, arguments, environment, and exit code are preserved. whyfail only observes; it never patches the running program.

whyfail --help
whyfail --version

3. The pytest plugin

No test rewrites needed:

pytest --whyfail

When a test fails, its diagnosis is printed alongside the normal failure output, with the failing test as context:

test_api.py::test_returns_user [call failed]
KeyError: 'user'

Failure analysis happens at the moment the exception is raised — inside pytest's own reporting hooks — so test frames and locals are still alive when the engine inspects them.

Supported diagnostics

Exception	What whyfail shows (from evidence)
KeyError	requested key, subject type, its keys, similar-key typo suggestions (only when a similar key actually exists)
IndexError	attempted index, sequence length, valid index range
TypeError	unsupported operands (from the message types + inspected operands), calling non-callables, subscripting non-subscriptables, argument-count mismatches, iterating non-iterables, None operands
AttributeError	object type, requested attribute, attributes that do exist, naming-mistake suggestions backed by a similar real attribute
NameError / UnboundLocalError	the missing name, scope, what is bound in that scope, bindings earlier/later in the source
ZeroDivisionError	the division expression and the runtime divisor when resolvable
ValueError	failed int()/float() conversions with the actual literal and runtime argument
ImportError / ModuleNotFoundError	missing module vs. missing symbol, importable parent prefixes, local shadowing files, circular-import wording
AssertionError	the asserted condition and the operand values that made it false; deliberate raise AssertionError is reported as such
anything else	an honest generic diagnosis — facts about the location, and an explicit insufficient evidence statement

Example output

IndexError: list index out of range
===================================

Likely cause
------------
Index 10 is out of range: the sequence has 3 element(s), so valid indexes are 0..2.

Runtime evidence
----------------
  - the indexed value is a list.
  - value: ['a', 'b', 'c']
  - length: 3
  - the failing access used the index 10.

Failure location
----------------
  app/main.py:6 in main()

  4 | def main():
  5 |     items = ["a", "b", "c"]
  6 |     return items[10]
    |            ^^^^^^^^^
Confidence: high

With insufficient evidence, the output says so instead:

Likely cause
------------
Insufficient runtime evidence to determine the root cause from the available
local evidence.

Architecture

Runtime failure
       │
       ▼
Failure Capture          capture.py   — exception chains + frames (live)
       │
       ▼
Evidence Collection      runtime.py   — bounded, guarded value inspection
       │                  redact.py    — conservative secret redaction
       ▼
Context Analysis         source.py    — source reading, AST ops, carets
       │
       ▼
Diagnosis Engine         engine.py + per-exception analyzers
       │
       ▼
Diagnostic Model         models.py    — structured data, never raw values
       │
       ├── CLI renderer      renderer.py / cli.py
       ├── Python API        api.py
       └── Pytest renderer   pytest_plugin.py

The pipeline runs inside the failing process — an except block, an installed sys.excepthook (CLI child), or pytest's reporting hooks — so the exception's frames and locals are alive when inspected. Rendered diagnostics are plain text built from the structured model.

Offline capture (whyfail run)

For whyfail run python app.py, the CLI launches the program with an observing sys.excepthook. On an unhandled failure the child itself runs the engine while frames are alive, redacts, and writes the structured result to a temporary JSON sidecar that only the parent CLI reads and renders. Normal behavior — output, traceback, exit code — is untouched.

For whyfail run pytest, the CLI launches pytest with the whyfail plugin loaded; the plugin writes the same sidecar protocol and the CLI renders it. Running pytest --whyfail directly renders inline instead.

Privacy and local-only guarantees

• No network. Ever. The engine performs no I/O beyond reading local files (source) and the local import path. There is no telemetry, no external service, no analytics.
• No raw values leave the process. Values are summarized and redacted before being recorded; the sidecar contains only rendered text.
• Redaction is conservative. Variable names that suggest secrets and values with credential signatures are replaced with <redacted> before anything is displayed or stored. Mappings redact entries whose keys are sensitive, and context source lines that contain credential literals are masked (the failing source line is shown verbatim, exactly like Python's own tracebacks).
• Source lines are displayed like tracebacks. Showing the source context is the same behavior as traceback/pytest — code you wrote, in your terminal. If a secret literal appears on the failing line itself, treat it as you would any traceback.

Limitations

• whyfail provides evidence-based likely explanations, not guaranteed root-cause analysis. The cause section states what the evidence supports, at a stated confidence; speculation appears only under "Possible explanations".
• Analysis happens at the moment the exception is alive. If a program overrides sys.excepthook, whyfail run cannot capture that failure.
• The interpreter must be able to import whyfail (the CLI runs children with the interpreter it is installed into).
• A pathological object whose __repr__ loops forever without raising cannot be interrupted from inside the same process (standard for any diagnostic tool); all raising/misbehaving cases are fully guarded.
• Source files are read from disk at analysis time, so source edits made between a crash and the analysis can make code context stale.

Development

git clone https://github.com/Sam3360/whyfail
cd whyfail
pip install -e ".[dev]"
python -m pytest

Project layout

pyproject.toml           packaging / metadata / entry points
src/whyfail/             the package (engine, analyzers, renderers, CLI, plugin)
tests/                   unit + integration + false-positive tests
.github/workflows/ci.yml CI (Linux, Python 3.9–3.14, build check)

Testing

The suite covers every supported exception type, runtime inspection, source extraction and highlighting, exception chaining, confidence, redaction, hostile/recursive/huge values, the CLI (exit codes, output preservation, -m and pytest invocations), the pytest plugin, and — importantly — false-positive tests proving the engine refuses to invent causes when the evidence does not support them.

Releasing

python -m build
twine upload dist/*     # after review

License

MIT — see LICENSE.