llmdebug 3.0.0

Structured debug snapshots for LLM-assisted debugging

llmdebug

Structured debug snapshots for LLM-assisted debugging.

---

llmdebug captures failure-time evidence — exception details, prioritized stack frames, local variables, and execution context — as a machine-readable artifact that works for both humans and coding agents. The goal is to make the first failing run useful, rather than reconstructing state after the fact.

The Debugging Loop

Without structured evidence, a typical loop looks like:

fail → infer missing state → guess patch → rerun → repeat

With llmdebug, the loop becomes:

fail → read snapshot → ranked hypotheses → minimal patch → verify

Installation

pip install 'llmdebug[cli]'   # Recommended: pytest plugin + CLI
pip install llmdebug           # Core library + pytest plugin only
pip install 'llmdebug[mcp]'    # MCP server for IDE integration

Other extras: jupyter, toon, evals — see Configuration Reference.

Start Here

• Use the package: start with the quick start below, then the Configuration Reference, CLI Reference, and MCP Reference.
• Contribute to the package: read Contributing, Testing Guide, and Quality Map.
• Run evals locally or in Docker: start with the Eval Quickstart, then Eval Framework, Eval Configs, and the Local/Docker Eval Runbook.
• Find any command fast: Master Runbook — setup, quality, debug CLI, eval ops, analysis, and maintenance all in one place.
• Browse canonical docs first: Docs Index separates active product docs from dated research notes and analyses.

Quick Start

Pytest (automatic)

Failing tests automatically create .llmdebug/latest.json:

pytest                                  # Failures create .llmdebug/latest.json
llmdebug                                # View crash summary
llmdebug show --detail context          # Full context (git, env, repro command)
llmdebug diff                           # Compare latest vs previous

Decorator

from llmdebug import SnapshotConfig, debug_snapshot

cfg = SnapshotConfig(out_dir=".llmdebug", redaction_profile="ci")

@debug_snapshot(config=cfg)
def main():
    data = load_data()
    process(data)

Context Manager

from llmdebug import SnapshotConfig, snapshot_section

cfg = SnapshotConfig(out_dir=".llmdebug", redaction_profile="prod")

with snapshot_section("data_processing", config=cfg):
    result = transform(data)

More entry points (production hooks, web middleware, Jupyter) in the Configuration Reference.

Features

• Automatic capture — pytest plugin, decorator, context manager, production hooks, web middleware
• Rich snapshots — exception chain, prioritized frames, typed locals (array shapes, dtypes), source context
• Layered detail — crash (~2K tokens) → full (~5K) → context (~10K) disclosure levels
• CLI inspection — show, list, frames, diff, git-context, clean
• Shareable exports — llmdebug export generates single-file artifacts or redacted bundle reports for CI/team sharing
• MCP server — 9 tools for Claude Code, Cursor, and other MCP-capable editors
• Hypothesis engine — auto-generated ranked debugging hypotheses from snapshot patterns
• Privacy controls — PII redaction profiles (dev / ci / prod), pattern-based redaction
• Jupyter integration — cell-error banners + %llmdebug magic commands
• Compact formats — json_compact (~40% smaller) and toon (~50% smaller) for LLM context

CLI

llmdebug                                # Latest snapshot (crash detail)
llmdebug show --detail full             # All frames
llmdebug show --detail context          # Everything (git, env, repro)
llmdebug show --json --detail context   # JSON output
llmdebug list                           # List snapshots
llmdebug list --search "KeyError"       # Search indexed snapshot metadata
llmdebug diff                           # Compare latest vs previous
llmdebug export latest --format html    # Shareable HTML artifact
llmdebug export --bundle-dir reports    # Redacted CI-safe bundle with manifest + index
llmdebug clean -k 5                     # Keep 5 most recent

Level	Content	~Tokens
crash (default)	Exception + crash frame	~2K
full	All frames + traceback	~5K
context	Everything (repro, git, env, coverage)	~10K

Full reference: docs/cli-reference.md

MCP Server

pip install 'llmdebug[mcp]'
llmdebug-mcp  # Start the MCP server (stdio transport)

Tool	Description
llmdebug_diagnose	Concise crash summary optimized for LLM consumption
llmdebug_show	Full expanded JSON snapshot with detail level control
llmdebug_list	List available snapshots with metadata
llmdebug_frame	Detailed view of a specific stack frame
llmdebug_git_context	On-demand enhanced git metadata for crash triage
llmdebug_diff	Compare two snapshots to show what changed
llmdebug_rca_status	Show latest RCA state for a session
llmdebug_rca_history	Show RCA attempt history
llmdebug_rca_advance	Manually advance RCA state machine

Claude Code configuration (.mcp.json):

{
  "mcpServers": {
    "llmdebug": {
      "command": "llmdebug-mcp"
    }
  }
}

Full reference: docs/mcp-reference.md

Output

On failure, .llmdebug/latest.json stores a versioned DebugSession envelope:

{
  "schema_version": "2.0",
  "kind": "llmdebug.debug_session",
  "session": {
    "name": "test_training_step",
    "timestamp_utc": "2026-01-27T14:30:52Z"
  },
  "snapshot": {
    "exception": {
      "type": "ValueError",
      "message": "operands could not be broadcast together..."
    },
    "frames": [
      {
        "file": "training.py",
        "line": 42,
        "function": "train_step",
        "code": "output = model(x) + residual",
        "locals": {
          "x": {"shape": [32, 64], "dtype": "float32"},
          "residual": {"shape": [32, 128], "dtype": "float32"}
        }
      }
    ]
  }
}

Configuration

Prefer config=SnapshotConfig(...) for decorator/context-manager capture settings. Legacy flat kwargs still work temporarily but emit a DeprecationWarning. Pytest plugin settings can still be driven by environment variables:

LLMDEBUG_OUTPUT_FORMAT=json pytest          # Use pretty JSON
LLMDEBUG_REDACTION_PROFILE=ci pytest        # Use CI redaction profile
LLMDEBUG_INCLUDE_GIT=false pytest           # Disable git context

Full parameter reference, output formats, and redaction profiles: docs/configuration.md

Documentation

Document	Description
Docs Index	Canonical docs map, grouped by reference, guides, ops, and research notes
Configuration Reference	Parameters, env vars, output formats, API surface
CLI Reference	Full CLI command reference
MCP Reference	MCP server JSON schemas and parameters
Architecture	System layers, capture pipeline, and design decisions
Testing Guide	Test workflows, snapshot-driven debugging, and integration guidance
Troubleshooting	Common setup/runtime failures and recovery steps
Eval Quickstart	Fastest copy-paste path for eval container builds, local model serving, and baseline/native runs
Eval Framework	Canonical starting point for running and interpreting evals
Eval Configs	Ready-made eval templates and TOML config mapping
Local/Docker Eval Runbook	Canonical eval-ops guide for local model endpoints, Docker wrapper runs, and SWE-Bench scoring
Contributing	Development setup and quality gates
Quality Map	Which checks block package changes vs staged eval work

Dated research notes, experiment logs, and roadmap documents remain under docs/, but they are supporting material rather than the primary starting point for package usage or eval operations.

License

MIT

Authors

• Nicolas Schuler — [e-mail]
• Balint Mate
• Vincenzo Scotti
• Raffaela Mirandola