Skip to main content

llmdebug logo

llmdebug

Structured debug snapshots for LLM-assisted debugging.

PyPI Python CI License Downloads


llmdebug captures failure-time evidence — exception details, prioritized stack frames, local variables, and execution context — as a machine-readable artifact that works for both humans and coding agents. The goal is to make the first failing run useful, rather than reconstructing state after the fact.

The Debugging Loop

Without structured evidence, a typical loop looks like:

fail → infer missing state → guess patch → rerun → repeat

With llmdebug, the loop becomes:

fail → read snapshot → ranked hypotheses → minimal patch → verify

Installation

pip install 'llmdebug[cli]'   # Recommended: pytest plugin + CLI
pip install llmdebug           # Core library + pytest plugin only
pip install 'llmdebug[mcp]'    # MCP server for IDE integration

Other extras: jupyter, toon, evals — see Configuration Reference.

Start Here

Quick Start

Pytest (automatic)

Failing tests automatically create .llmdebug/latest.json:

pytest                                  # Failures create .llmdebug/latest.json
llmdebug                                # View crash summary
llmdebug show --detail context          # Full context (git, env, repro command)
llmdebug diff                           # Compare latest vs previous

Decorator

from llmdebug import SnapshotConfig, debug_snapshot

cfg = SnapshotConfig(out_dir=".llmdebug", redaction_profile="ci")

@debug_snapshot(config=cfg)
def main():
    data = load_data()
    process(data)

Context Manager

from llmdebug import SnapshotConfig, snapshot_section

cfg = SnapshotConfig(out_dir=".llmdebug", redaction_profile="prod")

with snapshot_section("data_processing", config=cfg):
    result = transform(data)

More entry points (production hooks, web middleware, Jupyter) in the Configuration Reference.

Features

  • Automatic capture — pytest plugin, decorator, context manager, production hooks, web middleware
  • Rich snapshots — exception chain, prioritized frames, typed locals (array shapes, dtypes), source context
  • Layered detail — crash (~2K tokens) → full (~5K) → context (~10K) disclosure levels
  • CLI inspection — show, list, frames, diff, git-context, clean
  • Shareable exports — llmdebug export generates single-file artifacts or redacted bundle reports for CI/team sharing
  • MCP server — 9 tools for Claude Code, Cursor, and other MCP-capable editors
  • Hypothesis engine — auto-generated ranked debugging hypotheses from snapshot patterns
  • Privacy controls — PII redaction profiles (dev / ci / prod), pattern-based redaction
  • Jupyter integration — cell-error banners + %llmdebug magic commands
  • Compact formats — json_compact (~40% smaller) and toon (~50% smaller) for LLM context

CLI

llmdebug                                # Latest snapshot (crash detail)
llmdebug show --detail full             # All frames
llmdebug show --detail context          # Everything (git, env, repro)
llmdebug show --json --detail context   # JSON output
llmdebug list                           # List snapshots
llmdebug list --search "KeyError"       # Search indexed snapshot metadata
llmdebug diff                           # Compare latest vs previous
llmdebug export latest --format html    # Shareable HTML artifact
llmdebug export --bundle-dir reports    # Redacted CI-safe bundle with manifest + index
llmdebug clean -k 5                     # Keep 5 most recent
Level Content ~Tokens
crash (default) Exception + crash frame ~2K
full All frames + traceback ~5K
context Everything (repro, git, env, coverage) ~10K

Full reference: docs/cli-reference.md

MCP Server

pip install 'llmdebug[mcp]'
llmdebug-mcp  # Start the MCP server (stdio transport)
Tool Description
llmdebug_diagnose Concise crash summary optimized for LLM consumption
llmdebug_show Full expanded JSON snapshot with detail level control
llmdebug_list List available snapshots with metadata
llmdebug_frame Detailed view of a specific stack frame
llmdebug_git_context On-demand enhanced git metadata for crash triage
llmdebug_diff Compare two snapshots to show what changed
llmdebug_rca_status Show latest RCA state for a session
llmdebug_rca_history Show RCA attempt history
llmdebug_rca_advance Manually advance RCA state machine

Claude Code configuration (.mcp.json):

{
  "mcpServers": {
    "llmdebug": {
      "command": "llmdebug-mcp"
    }
  }
}

Full reference: docs/mcp-reference.md

Output

On failure, .llmdebug/latest.json stores a versioned DebugSession envelope:

{
  "schema_version": "2.0",
  "kind": "llmdebug.debug_session",
  "session": {
    "name": "test_training_step",
    "timestamp_utc": "2026-01-27T14:30:52Z"
  },
  "snapshot": {
    "exception": {
      "type": "ValueError",
      "message": "operands could not be broadcast together..."
    },
    "frames": [
      {
        "file": "training.py",
        "line": 42,
        "function": "train_step",
        "code": "output = model(x) + residual",
        "locals": {
          "x": {"shape": [32, 64], "dtype": "float32"},
          "residual": {"shape": [32, 128], "dtype": "float32"}
        }
      }
    ]
  }
}

Configuration

Prefer config=SnapshotConfig(...) for decorator/context-manager capture settings. Legacy flat kwargs still work temporarily but emit a DeprecationWarning. Pytest plugin settings can still be driven by environment variables:

LLMDEBUG_OUTPUT_FORMAT=json pytest          # Use pretty JSON
LLMDEBUG_REDACTION_PROFILE=ci pytest        # Use CI redaction profile
LLMDEBUG_INCLUDE_GIT=false pytest           # Disable git context

Full parameter reference, output formats, and redaction profiles: docs/configuration.md

Documentation

Document Description
Docs Index Canonical docs map, grouped by reference, guides, ops, and research notes
Configuration Reference Parameters, env vars, output formats, API surface
CLI Reference Full CLI command reference
MCP Reference MCP server JSON schemas and parameters
Architecture System layers, capture pipeline, and design decisions
Testing Guide Test workflows, snapshot-driven debugging, and integration guidance
Troubleshooting Common setup/runtime failures and recovery steps
Eval Quickstart Fastest copy-paste path for eval container builds, local model serving, and baseline/native runs
Eval Framework Canonical starting point for running and interpreting evals
Eval Configs Ready-made eval templates and TOML config mapping
Local/Docker Eval Runbook Canonical eval-ops guide for local model endpoints, Docker wrapper runs, and SWE-Bench scoring
Contributing Development setup and quality gates
Quality Map Which checks block package changes vs staged eval work

Dated research notes, experiment logs, and roadmap documents remain under docs/, but they are supporting material rather than the primary starting point for package usage or eval operations.

License

MIT

Authors

Metadata

Release files for llmdebug 3.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for llmdebug 3.0.0
File Size Uploaded Provenance Actions
llmdebug-3.0.0.tar.gz 7.7 MB Without provenance Details for llmdebug-3.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for llmdebug 3.0.0
File Interpreter ABI Platform Actions
llmdebug-3.0.0-py3-none-any.whl Python 3 none any Details for llmdebug-3.0.0-py3-none-any.whl

Total release size: 7.9 MB

Release files / llmdebug-3.0.0.tar.gz

Download URL llmdebug-3.0.0.tar.gz
Size 7.7 MB
Tags Source
SHA-256 checksum
How to use checksums
c9a3236c26daea08bb9ba3cd637e425f6108cc83908658798237e9eef5d03d3a
BLAKE2b-256 checksum
How to use checksums
6f77027a02560bab2e20b4916154b7f81749598388c1515f1947e05d0efca1d7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.11.3 {"installer":{"name":"uv","version":"0.11.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / llmdebug-3.0.0-py3-none-any.whl

Download URL llmdebug-3.0.0-py3-none-any.whl
Size 146.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
69d02ae86c3dce9857a0a01ddb04206588f587f3e19c037be80bfb2d3353cc57
BLAKE2b-256 checksum
How to use checksums
1ef01312a23bb7a3289482d90c65c217de6f98a3c81eab8158027a381aff45d6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.11.3 {"installer":{"name":"uv","version":"0.11.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for llmdebug 3.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

3.0.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page