llmdebug
Structured debug snapshots for LLM-assisted debugging.
llmdebug captures failure-time evidence — exception details, prioritized stack
frames, local variables, and execution context — as a machine-readable artifact
that works for both humans and coding agents. The goal is to make the first
failing run useful, rather than reconstructing state after the fact.
The Debugging Loop
Without structured evidence, a typical loop looks like:
fail → infer missing state → guess patch → rerun → repeat
With llmdebug, the loop becomes:
fail → read snapshot → ranked hypotheses → minimal patch → verify
Installation
pip install 'llmdebug[cli]' # Recommended: pytest plugin + CLI
pip install llmdebug # Core library + pytest plugin only
pip install 'llmdebug[mcp]' # MCP server for IDE integration
Other extras: jupyter, toon, evals — see Configuration Reference.
Start Here
- Use the package: start with the quick start below, then the Configuration Reference, CLI Reference, and MCP Reference.
- Contribute to the package: read Contributing, Testing Guide, and Quality Map.
- Run evals locally or in Docker: start with the Eval Quickstart, then Eval Framework, Eval Configs, and the Local/Docker Eval Runbook.
- Find any command fast: Master Runbook — setup, quality, debug CLI, eval ops, analysis, and maintenance all in one place.
- Browse canonical docs first: Docs Index separates active product docs from dated research notes and analyses.
Quick Start
Pytest (automatic)
Failing tests automatically create .llmdebug/latest.json:
pytest # Failures create .llmdebug/latest.json
llmdebug # View crash summary
llmdebug show --detail context # Full context (git, env, repro command)
llmdebug diff # Compare latest vs previous
Decorator
from llmdebug import SnapshotConfig, debug_snapshot
cfg = SnapshotConfig(out_dir=".llmdebug", redaction_profile="ci")
@debug_snapshot(config=cfg)
def main():
data = load_data()
process(data)
Context Manager
from llmdebug import SnapshotConfig, snapshot_section
cfg = SnapshotConfig(out_dir=".llmdebug", redaction_profile="prod")
with snapshot_section("data_processing", config=cfg):
result = transform(data)
More entry points (production hooks, web middleware, Jupyter) in the Configuration Reference.
Features
- Automatic capture — pytest plugin, decorator, context manager, production hooks, web middleware
- Rich snapshots — exception chain, prioritized frames, typed locals (array shapes, dtypes), source context
- Layered detail —
crash(~2K tokens) →full(~5K) →context(~10K) disclosure levels - CLI inspection —
show,list,frames,diff,git-context,clean - Shareable exports —
llmdebug exportgenerates single-file artifacts or redacted bundle reports for CI/team sharing - MCP server — 9 tools for Claude Code, Cursor, and other MCP-capable editors
- Hypothesis engine — auto-generated ranked debugging hypotheses from snapshot patterns
- Privacy controls — PII redaction profiles (
dev/ci/prod), pattern-based redaction - Jupyter integration — cell-error banners +
%llmdebugmagic commands - Compact formats —
json_compact(~40% smaller) andtoon(~50% smaller) for LLM context
CLI
llmdebug # Latest snapshot (crash detail)
llmdebug show --detail full # All frames
llmdebug show --detail context # Everything (git, env, repro)
llmdebug show --json --detail context # JSON output
llmdebug list # List snapshots
llmdebug list --search "KeyError" # Search indexed snapshot metadata
llmdebug diff # Compare latest vs previous
llmdebug export latest --format html # Shareable HTML artifact
llmdebug export --bundle-dir reports # Redacted CI-safe bundle with manifest + index
llmdebug clean -k 5 # Keep 5 most recent
| Level | Content | ~Tokens |
|---|---|---|
crash (default) |
Exception + crash frame | ~2K |
full |
All frames + traceback | ~5K |
context |
Everything (repro, git, env, coverage) | ~10K |
Full reference: docs/cli-reference.md
MCP Server
pip install 'llmdebug[mcp]'
llmdebug-mcp # Start the MCP server (stdio transport)
| Tool | Description |
|---|---|
llmdebug_diagnose |
Concise crash summary optimized for LLM consumption |
llmdebug_show |
Full expanded JSON snapshot with detail level control |
llmdebug_list |
List available snapshots with metadata |
llmdebug_frame |
Detailed view of a specific stack frame |
llmdebug_git_context |
On-demand enhanced git metadata for crash triage |
llmdebug_diff |
Compare two snapshots to show what changed |
llmdebug_rca_status |
Show latest RCA state for a session |
llmdebug_rca_history |
Show RCA attempt history |
llmdebug_rca_advance |
Manually advance RCA state machine |
Claude Code configuration (.mcp.json):
{
"mcpServers": {
"llmdebug": {
"command": "llmdebug-mcp"
}
}
}
Full reference: docs/mcp-reference.md
Output
On failure, .llmdebug/latest.json stores a versioned DebugSession envelope:
{
"schema_version": "2.0",
"kind": "llmdebug.debug_session",
"session": {
"name": "test_training_step",
"timestamp_utc": "2026-01-27T14:30:52Z"
},
"snapshot": {
"exception": {
"type": "ValueError",
"message": "operands could not be broadcast together..."
},
"frames": [
{
"file": "training.py",
"line": 42,
"function": "train_step",
"code": "output = model(x) + residual",
"locals": {
"x": {"shape": [32, 64], "dtype": "float32"},
"residual": {"shape": [32, 128], "dtype": "float32"}
}
}
]
}
}
Configuration
Prefer config=SnapshotConfig(...) for decorator/context-manager capture
settings. Legacy flat kwargs still work temporarily but emit a
DeprecationWarning. Pytest plugin settings can still be driven by environment
variables:
LLMDEBUG_OUTPUT_FORMAT=json pytest # Use pretty JSON
LLMDEBUG_REDACTION_PROFILE=ci pytest # Use CI redaction profile
LLMDEBUG_INCLUDE_GIT=false pytest # Disable git context
Full parameter reference, output formats, and redaction profiles: docs/configuration.md
Documentation
| Document | Description |
|---|---|
| Docs Index | Canonical docs map, grouped by reference, guides, ops, and research notes |
| Configuration Reference | Parameters, env vars, output formats, API surface |
| CLI Reference | Full CLI command reference |
| MCP Reference | MCP server JSON schemas and parameters |
| Architecture | System layers, capture pipeline, and design decisions |
| Testing Guide | Test workflows, snapshot-driven debugging, and integration guidance |
| Troubleshooting | Common setup/runtime failures and recovery steps |
| Eval Quickstart | Fastest copy-paste path for eval container builds, local model serving, and baseline/native runs |
| Eval Framework | Canonical starting point for running and interpreting evals |
| Eval Configs | Ready-made eval templates and TOML config mapping |
| Local/Docker Eval Runbook | Canonical eval-ops guide for local model endpoints, Docker wrapper runs, and SWE-Bench scoring |
| Contributing | Development setup and quality gates |
| Quality Map | Which checks block package changes vs staged eval work |
Dated research notes, experiment logs, and roadmap documents remain under
docs/, but they are supporting material rather than the primary starting
point for package usage or eval operations.
License
MIT
Authors
- Nicolas Schuler — schuler.nicolas@proton.me
- Balint Mate
- Vincenzo Scotti
- Raffaela Mirandola
Metadata
Release files for llmdebug 3.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| llmdebug-3.0.0.tar.gz | 7.7 MB | Without provenance | Details for llmdebug-3.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| llmdebug-3.0.0-py3-none-any.whl | Python 3 | none | any | Details for llmdebug-3.0.0-py3-none-any.whl |
Total release size: 7.9 MB
Release files / llmdebug-3.0.0.tar.gz
| Download URL | llmdebug-3.0.0.tar.gz |
|---|---|
| Size | 7.7 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c9a3236c26daea08bb9ba3cd637e425f6108cc83908658798237e9eef5d03d3a
|
|
BLAKE2b-256 checksum How to use checksums |
6f77027a02560bab2e20b4916154b7f81749598388c1515f1947e05d0efca1d7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.11.3 {"installer":{"name":"uv","version":"0.11.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / llmdebug-3.0.0-py3-none-any.whl
| Download URL | llmdebug-3.0.0-py3-none-any.whl |
|---|---|
| Size | 146.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
69d02ae86c3dce9857a0a01ddb04206588f587f3e19c037be80bfb2d3353cc57
|
|
BLAKE2b-256 checksum How to use checksums |
1ef01312a23bb7a3289482d90c65c217de6f98a3c81eab8158027a381aff45d6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.11.3 {"installer":{"name":"uv","version":"0.11.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for llmdebug 3.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.