whatbroke 0.1.0

Snapshot your project's environment over time and find out what actually broke it.

whatbroke

It worked yesterday. Why is it broken today?

whatbroke records your project's environment over time — dependencies, lockfiles, env vars, config files, container base images, runtime versions — so that when something breaks you can see exactly what moved, instead of guessing.

30 seconds

A real session — werkzeug 3.0 removed werkzeug.urls.url_quote, which broke a great many Flask apps in 2023. Reproduce it yourself with scripts/record-demo.sh.

$ whatbroke snapshot --label "green: callbacks working"
Snapshot #1 captured (126 entries)

# ...a week later, a dependency floats and the app stops working:

$ python app.py
Traceback (most recent call last):
  File "app.py", line 1, in <module>
    from werkzeug.urls import url_quote
ImportError: cannot import name 'url_quote' from 'werkzeug.urls'

$ whatbroke snapshot --label "red: ImportError"
$ whatbroke diff
Diff: #1 (green: callbacks working) -> #2 (red: ImportError)

Python dependencies  (1 changed)
  ~ werkzeug  2.2.3 -> 3.0.1

Config files  (1 changed)
  ~ docker-compose.yml  content changed (+1/-1 lines)  sha256:19325d3ea1f8

Nothing in the source changed. The diff finds it in one command.

Quickstart

Four commands, under a minute:

pipx install whatbroke     # or: pip install whatbroke
cd your-project
whatbroke init             # detects your package managers, writes a config
whatbroke snapshot         # capture the environment as it is right now

Then later, once something has broken:

whatbroke snapshot
whatbroke diff             # what changed between the last two snapshots

Diff: #1 (working state, 7 minutes ago) → #2 (broken state, just now)

Python dependencies  (2 changed, 1 added, 1 removed)
  ~ requests  2.31.0 → 2.32.3
  ~ urllib3   2.0.7 → 2.2.1
  + gunicorn  21.2.0
  - flask     3.0.0

Config files  (3 changed)
  ~ Dockerfile          content changed (+1/-1 lines)  sha256:843cf881543a
  ~ docker-compose.yml  content changed (+2/-2 lines)  sha256:3293327d0668
  ~ requirements.txt    content changed (+3/-3 lines)  sha256:4270126a4293

Container base images  (1 changed)
  ~ Dockerfile#0 (builder)  python:3.11-slim → python:3.12-slim

To capture history automatically from here on:

whatbroke install-hook     # snapshot on every commit

Why this exists

The failure mode is familiar. A test that passed on Friday fails on Monday. You didn't change anything. git log is clean. And yet.

Something did change — just not in your source. A transitive dependency floated to a new minor. A Docker base image tag moved under you. A teammate added an env var to the deploy config. Your colleague's machine has a different Python patch version.

git bisect can't help, because git doesn't track any of it. Version control gave us a rigorous answer to "what changed in the code?" and nothing comparable for "what changed around the code?". whatbroke is that missing history.

When the diff is long

Hand it the error and let Claude rank the suspects:

export ANTHROPIC_API_KEY=sk-ant-...
whatbroke explain --error "$(cat error.log)"

The diff plausibly explains this error.

1. urllib3 2.2.1 -> 2.0.7  [high]  python-deps
   why    urllib3 2.0 predates the Retry re-export the traceback imports.
   check  pip install 'urllib3==2.2.1' and re-run the failing import

2. requests 2.31.0 -> 2.32.3  [medium]  python-deps
   why    requests 2.32 widened its urllib3 pin, likely pulling the older one in.
   check  pip install 'requests==2.31.0' and see if pip moves urllib3 with it

It is instructed to say so when the diff doesn't explain your error, rather than inventing a plausible-sounding culprit.

Everything except explain works offline. No API key, no account, no telemetry.

When you have a repro command

Let it binary-search your history for the breaking change:

whatbroke bisect --command "pytest tests/test_api.py" --yes

Verifying the good and bad ends behave as claimed...
  testing  #8 (bad end)   FAIL  exit 1
  testing  #1 (good end)  PASS  exit 0
  testing  #4             PASS  exit 0
  testing  #6             FAIL  exit 1
  testing  #5             FAIL  exit 1

Environment restored to snapshot #9

First bad snapshot: #5   (last good was #4)

...followed by the diff between those two snapshots.

⚠️ bisect modifies your environment

To test an old snapshot it has to become that snapshot: it checks out old commits and reinstalls dependencies. Run it in a container or a disposable venv.

• Requires --yes. Use --dry-run to print the exact plan and change nothing.
• Refuses to start on a dirty working tree, naming the files at risk.
• Takes a restore point snapshot first and restores to it afterwards — including on Ctrl-C — returning you to your branch, not a detached HEAD.
• Warns if whatbroke is installed into the environment it's about to modify.
• A snapshot it can't restore is skipped, not blamed. If skips leave a gap it reports the remaining candidates rather than guessing between them.

If a run is interrupted, recover with whatbroke restore <id> --yes.

Commands

whatbroke init	Scaffold .whatbroke/config.toml, detecting your package managers
whatbroke snapshot [-l LABEL]	Capture the current environment
whatbroke list	Snapshots, newest first
whatbroke show <snap>	Dump one snapshot
whatbroke diff [A] [B]	What changed (defaults to the last two)
whatbroke explain --error "..."	Rank the changes by likely blame, via Claude
whatbroke bisect --command "..." --yes	Binary-search for the breaking change
whatbroke restore <snap> --yes	Put the environment back
whatbroke install-hook	Snapshot automatically on every commit
whatbroke prune --older-than 30d --yes	Bound the store's size

Snapshot references accept ids or shortcuts: latest, latest~2, oldest, -3. Add --json to diff, explain and bisect for machine-readable output, and --category to narrow to one kind of change.

What gets captured

Python	poetry.lock, pdm.lock, uv.lock, requirements.txt, or pip freeze
Node	package-lock.json, pnpm-lock.yaml, yarn.lock — resolved versions, not a file hash
Rust / Go	Cargo.lock, go.sum
Env vars	Names always; values only where you allow-list them
Config files	Contents + hash for files you list, secrets scrubbed
Containers	FROM lines, with the real image digest when Docker is available
System	OS, architecture, and versions of the runtimes this project actually uses

Everything lives in .whatbroke/store.db (SQLite — see SCHEMA.md). whatbroke init adds it to your .gitignore; keep it there, because a committed store gets overwritten by an ordinary git checkout.

Git, Docker and the various package managers are all optional. whatbroke uses what it finds and tells you what it skipped.

Configuration

.whatbroke/config.toml, generated by whatbroke init:

[env]
capture_values = ["NODE_ENV", "CI", "PORT"]
ignore = ["TMP*"]
extra_secret_patterns = []

[files]
track = [".env.example", "docker-compose.yml", "tsconfig.json"]
max_bytes = 262144

[python]
source = "auto"   # auto | poetry | pdm | uv | requirements | pip-freeze | none

[ai]
model = "claude-sonnet-5"   # WHATBROKE_MODEL overrides this

Secrets

whatbroke treats "never write a secret to disk" as a hard requirement, not a best effort:

• Env var names are always recorded; values never are unless you explicitly allow-list the name.
• A name matching a secret pattern (*_KEY, *_TOKEN, *_SECRET, *PASSWORD*, DATABASE_URL, …) is never captured. Allow-listing it does not override this — that is deliberate and not configurable.
• A value that looks like a credential (sk-…, ghp_…, a JWT, a PEM key, a URL with inline credentials) is withheld regardless of its name.
• Tracked config file contents are scrubbed of secret-looking assignments before storage.
• Withheld values are recorded as a salted hash, with a random salt per store — so a diff can say "DATABASE_URL changed" while the digest is worthless to anyone who obtains the file.
• whatbroke explain sends the diff to the Anthropic API. Because the store never holds a secret, the request cannot leak one.

Found a way around this? Please report it privately — see CONTRIBUTING.md.

Requirements

Python 3.11+. Linux, macOS and Windows.

Contributing

Bug reports from real broken environments are the most useful thing you can send. New ecosystem collectors are the easiest code contribution — see CONTRIBUTING.md.

pip install -e ".[dev]"
pytest

License

MIT — see LICENSE.