breadcrumb 0.2.12

An automatic, non-intrusive execution tracer for Python for iterative IDE-less debugging and code-flow analysis

🍞 Breadcrumb: Automatic Execution Tracing

Breadcrumb lets you do this:

# Run your code with automatic tracing (NO code changes needed!)
breadcrumb run -c myproject --timeout 60 python my_code.py

Output:

============================================================
BREADCRUMB RUN REPORT
============================================================

Key Metrics:
  Total Events: 22
  - Calls: 9
  - Returns: 9
  - Exceptions: 0
  Duration: 4.06 ms
  Status: completed

Call Tree:
-> __main__.<module>
  -> __main__.main
    -> __main__.add
    <- __main__.add (0.05ms)
    -> __main__.multiply
    <- __main__.multiply (0.05ms)
    -> __main__.greet
    <- __main__.greet (0.04ms)
    -> __main__.calculate_total
      -> __main__.multiply
      <- __main__.multiply (0.08ms)
      -> __main__.multiply
      <- __main__.multiply (0.08ms)
      -> __main__.add
      <- __main__.add (0.04ms)
    <- __main__.calculate_total (0.51ms)
  <- __main__.main (1.46ms)
<- __main__.<module> (1.61ms)

Top 10 Most Called Functions:
    __main__.multiply: 3 calls
    __main__.add: 2 calls
    __main__.greet: 1 calls
    __main__.<module>: 1 calls
    __main__.main: 1 calls
    __main__.calculate_total: 1 calls

Recommendations on what to do next:
  Detailed Flow: breadcrumb query -c example01 --flow
  Untraced calls: breadcrumb query -c example01 --gaps
  View specific call: breadcrumb query -c example01 --call <call_id>
     Example 'multiply': breadcrumb query -c example01 --call multiply

Dig in:

# Query critical debug information: Called by, calls made, performance stats, input args, return values
breadcrumb query -c example01 --call multiply

Output:

{
  "function": "__main__.multiply",
  "calls": [
    {
      "function": "__main__.multiply",
      "timestamp": "2025-10-11T22:50:39.444263",
      "args": {
        "a": 7,
        "b": 6
      },
      "return_value": 42,
      "duration_ms": 0.049,
      "called_by": "__main__.main",
      "calls_made": []
    },
    {
      "function": "__main__.multiply",
      "timestamp": "2025-10-11T22:50:39.444732",
      "args": {
        "a": 100,
        "b": 2
      },
      "return_value": 200,
      "duration_ms": 0.082,
      "called_by": "__main__.calculate_total",
      "calls_made": []
    },
    {
      "function": "__main__.multiply",
      "timestamp": "2025-10-11T22:50:39.444900",
      "args": {
        "a": 200,
        "b": 0.15
      },
      "return_value": 30.0,
      "duration_ms": 0.083,
      "called_by": "__main__.calculate_total",
      "calls_made": []
    }
  ]
}

# Find out what was called by traced calls that isn't yet traced by breadcrumb
breadcrumb query -c example01 --gaps

Output:

{
  "untraced_calls": [
    {
      "function": "json.dumps",
      "module": "json",
      "called_by": "__main__.format_response",
      "call_count": 3,
      "suggested_include": "json.*"
    },
    {
      "function": "requests.post",
      "module": "requests",
      "called_by": "__main__.send_webhook",
      "call_count": 1,
      "suggested_include": "requests.*"
    }
  ],
  "tip": "Add these to your config to trace them:\n  breadcrumb config edit myapp --add-include 'json.*'\n  breadcrumb config edit myapp --add-include 'requests.*'"
}

Expand tracing and re-run:

# include untraced calls
breadcrumb config edit --add-include 'requests.*'
# run again
breadcrumb run python my_code.py
breadcrumb query --call post

Output:

{
  "function": "requests.post",
  "calls": [
    {
      "timestamp": "2025-10-11T15:30:42.123",
      "args": {"url": "https://api.example.com/webhook", "data": {"event": "user.signup"}},
      "return_value": {"status": 200, "body": "OK"},
      "duration_ms": 234.5,
      "called_by": "__main__.send_webhook",
      "calls_made": ["json.dumps"]
    }
  ]
}

---

Why Would You Need This?

You probably won't. But your AI coding agent will LOVE this.

The Problem: AI Agents Can't "Step Through" Code

When humans debug, we:

• Set breakpoints and step through execution
• Watch variables change in real-time
• See the actual call stack in our debugger
• Understand what the code actually did vs what we thought it would do

When AI agents debug, they:

• ❌ Can't set breakpoints (no interactive debugger access)
• ❌ Can't "watch" execution (no real-time visibility)
• ❌ Can't inspect the call stack (no execution context)
• ❌ Must guess what happened by reading static code

Result: AI agents spend 80% of debugging time making guesses, reading logs, and asking "what actually ran?"

The Solution: Breadcrumb = X-Ray Vision for AI Agents

Breadcrumb captures what actually happened during execution:

• ✅ Every function call with arguments and return values
• ✅ Exact execution flow in chronological order
• ✅ Gaps in coverage showing untraced function calls
• ✅ Call relationships (who called what)
• ✅ Performance data (execution duration)

No code changes. No manual logging. No guessing.

Just run your code with breadcrumb run and query the execution trace with structured commands.

---

Codex CLI using breadcrumb

# Prompt
By running 'breadcrumb', present a table showing how long each mathematical function call in the example took.

Codex CLI using advanced queries for complex tasks

Key Features

🎯 Gap Detection

Problem: You can't trace EVERYTHING (performance overhead). So what aren't you tracing?

Solution: breadcrumb query --gaps shows you exactly which functions were called but not traced.

breadcrumb query -c myapp --gaps
# Shows: requests.post called 5 times by myapp.api_client
# Suggests: breadcrumb config edit myapp --add-include 'requests.*'

Workflow: Start minimal → discover gaps → expand tracing → repeat

This is iterative debugging - trace only what you need, when you need it.

🤖 AI-FIRST Design

Traditional tools: Optimize for human convenience (implicit context, smart defaults) Breadcrumb: Optimizes for AI reliability (explicit context, zero ambiguity)

# ❌ IMPLICIT (confusing for AI agents)
breadcrumb run python script.py          # Which database? Which config?
breadcrumb query --gaps                   # Where are the traces?

# ✅ EXPLICIT (perfect for AI agents)
breadcrumb run -c myapp python script.py  # Clear: using myapp config
breadcrumb query -c myapp --gaps          # Clear: querying myapp database

Result: Zero context confusion across sessions. AI agents can resume debugging workflows perfectly.

📊 Smart Queries (No SQL Required)

AI agents don't need to learn SQL. They just need structured data.

# Find untraced calls
breadcrumb query -c myapp --gaps

# Show function I/O
breadcrumb query -c myapp --call send_webhook

# Show execution timeline
breadcrumb query -c myapp --flow

# Filter by module
breadcrumb query -c myapp --flow --module myapp

# Search captured events for a literal string (function names, args, payloads, etc.)
breadcrumb query -c myapp --fuzzy breadcrumbs

All output is structured JSON, ready for AI consumption.

Call queries accept substrings, so breadcrumb query -c myapp --call Pizza resolves to __main__.Pizza (and its __init__) and breadcrumb query -c myapp --call "__main__." returns every function in __main__. Constructors capture their keyword arguments and return values automatically, making object creation auditable without extra logging.

Need a human-readable table? Switch formats:

# Table output with full values preserved for wide SQL results
breadcrumb --format table query -c myapp --disable-truncation \
  "SELECT function_name, data FROM trace_events ORDER BY timestamp DESC LIMIT 5"

# Increase captured payload size when recording new traces (default 2000 chars)
breadcrumb run -c myapp -t 60 --max-chars 5000 python main.py

🚫 Zero Code Changes

# ✅ Just run with breadcrumb
breadcrumb run -c myapp --timeout 60 python main.py
# NO code modifications needed!

Your codebase stays clean. Tracing is external.

🔒 Automatic Secret Redaction

Breadcrumb automatically detects and redacts 50+ sensitive patterns:

• Passwords, API keys, tokens
• Credit cards, SSNs
• JWTs, AWS keys, GitHub tokens

data = {"api_key": "sk-1234567890", "password": "secret"}
# Stored as: {"api_key": "[REDACTED]", "password": "[REDACTED]"}

Safe for production traces.

⚡ Low Overhead

• PEP 669 backend (Python 3.12+): < 2% overhead
• sys.settrace backend (Python 3.8+): < 5% overhead
• Selective tracing: Only trace what you need
• Async writes: Non-blocking trace persistence

---

Quick Start

Installation

# Using uv (recommended)
uv pip install -e .

# Or using pip
pip install -e .

The Complete Workflow

1️⃣ Initialize your project (one-time setup):

breadcrumb init myapp

Creates:

• ~/.breadcrumb/myapp.yaml (config with sensible defaults)
• ~/.breadcrumb/myapp-traces.duckdb (trace database)

2️⃣ Run your code with tracing:

breadcrumb run -c myapp --timeout 60 python main.py

Automatically traces execution. NO code changes needed.

3️⃣ Find gaps in coverage:

breadcrumb query -c myapp --gaps

Shows which functions were called but not traced.

4️⃣ Expand tracing based on gaps:

breadcrumb config edit myapp --add-include 'requests.*'
breadcrumb config edit myapp --add-include 'sqlalchemy.*'

5️⃣ Re-run with expanded tracing:

breadcrumb run -c myapp --timeout 60 python main.py

Now traces additional modules!

6️⃣ Query function details:

breadcrumb query -c myapp --call my_function
breadcrumb query -c myapp --call "__main__."    # Substring match, includes constructors

Shows args, returns, duration, caller/callee relationships.

7️⃣ View execution timeline:

breadcrumb query -c myapp --flow

Shows chronological execution order with call stack depth.

8️⃣ (Optional) Capture richer payloads:

breadcrumb run -c myapp --timeout 60 --max-chars 5000 python main.py

Increases argument/return capture size without touching your source code.

---

AI-FIRST Design Philosophy

Why -c PROJECT is REQUIRED

All commands require explicit project specification:

breadcrumb run -c myapp python script.py       # REQUIRED
breadcrumb query -c myapp --gaps               # REQUIRED
breadcrumb config edit myapp --add-include 'x' # REQUIRED

This eliminates:

• ❌ "Which database am I querying?"
• ❌ "Which config is active?"
• ❌ "Where did my traces go?"
• ❌ Context loss between AI agent sessions

Why it matters:

• Humans: "Ugh, typing -c myapp every time is annoying!"
• AI Agents: "Perfect! No ambiguity, no context confusion!"

Design philosophy: Explicit > Implicit for AI workflows.

AI agents don't care about typing extra characters. They care about reliability across sessions.

---

Command Reference

Core Commands

# Initialize project
breadcrumb init <project-name>

# Run with tracing
breadcrumb run -c <project> --timeout <seconds> <command>

# Smart queries
breadcrumb query -c <project> --gaps           # Find untraced calls
breadcrumb query -c <project> --call <func>    # Show function I/O
breadcrumb query -c <project> --flow           # Show execution timeline

# Inspect recorded traces
breadcrumb list -c <project>
breadcrumb get -c <project> <trace_id>
breadcrumb report -c <project>
breadcrumb top -c <project> 20
breadcrumb exceptions -c <project> --since 2h
breadcrumb clear -c <project> --force

# Config management
breadcrumb config create <name>
breadcrumb config edit <name> --add-include <pattern>
breadcrumb config edit <name> --remove-include <pattern>
breadcrumb config show <name>
breadcrumb config list

Query Examples

# Find what's not traced
breadcrumb query -c myapp --gaps

# Show function with args/returns
breadcrumb query -c myapp --call process_payment

# Show execution flow for module
breadcrumb query -c myapp --flow --module myapp

# Substring call search (returns constructors too)
breadcrumb query -c myapp --call "__main__."       # All __main__ functions
breadcrumb query -c myapp --call Pizza             # Matches __main__.Pizza/__init__

# Raw SQL query (still supported)
breadcrumb query -c myapp --sql "SELECT * FROM traces WHERE status='failed'"

Config Examples

# Start with minimal tracing
breadcrumb init myapp
# Creates config with include: ['__main__']

# Expand based on gaps
breadcrumb config edit myapp --add-include 'requests.*'
breadcrumb config edit myapp --add-include 'sqlalchemy.*'
breadcrumb config edit myapp --add-include 'myapp.services.*'

# Remove noisy modules
breadcrumb config edit myapp --remove-include 'logging.*'

# View current config
breadcrumb config show myapp

---

Use Cases for AI Coding Agents

1. Understanding Legacy Code

Human approach:

• Read docs (if they exist)
• Step through with debugger
• Ask teammates

AI agent approach:

# Run the code
breadcrumb run -c legacy --timeout 120 python legacy_system.py

# See what actually executed
breadcrumb query -c legacy --flow

# Find entry points
breadcrumb query -c legacy --gaps

Result: AI agent sees actual execution flow, not just static code.

2. Debugging API Integration Issues

Human approach:

• Add print statements
• Check API logs
• Use network inspector

AI agent approach:

# Trace the integration
breadcrumb config edit myapp --add-include 'requests.*'
breadcrumb run -c myapp --timeout 60 python test_api.py

# See exact API calls
breadcrumb query -c myapp --call 'requests.post'

Result: AI agent sees exact request args, response values, and timing.

3. Performance Investigation

Human approach:

• Use profiler
• Add timing code
• Analyze bottlenecks

AI agent approach:

# Run with tracing
breadcrumb run -c myapp --timeout 300 python load_test.py

# Find slow functions
breadcrumb query -c myapp --sql "SELECT function_name, AVG(duration_ms) FROM trace_events GROUP BY function_name ORDER BY AVG(duration_ms) DESC LIMIT 10"

Result: AI agent identifies performance bottlenecks from real execution data.

4. Test Failure Analysis

Human approach:

• Re-run test
• Add debug output
• Check assertions

AI agent approach:

# Run failing test with tracing
breadcrumb run -c tests --timeout 60 pytest tests/test_payment.py::test_refund

# See what actually happened
breadcrumb query -c tests --call process_refund

# Check execution flow
breadcrumb query -c tests --flow --module myapp.payments

Result: AI agent sees exact execution path that led to failure.

---

MCP Integration (Claude Desktop)

Breadcrumb includes an MCP server for Claude Desktop integration.

Setup:

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "breadcrumb": {
      "command": "breadcrumb",
      "args": ["serve-mcp"],
      "env": {}
    }
  }
}

Available Tools:

• breadcrumb__query_traces - Execute SQL queries
• breadcrumb__get_trace - Get trace details by ID
• breadcrumb__find_exceptions - Find exceptions in traces
• breadcrumb__analyze_performance - Analyze function performance

Usage:

Tell Claude: "Use breadcrumb to find the last exception in my app"
Claude will use: breadcrumb__find_exceptions

---

Advanced Configuration

Python API (Optional)

If you need programmatic control, you can still use the Python API:

import breadcrumb

# Basic configuration
breadcrumb.init(
    enabled=True,
    sample_rate=1.0,
    db_path="~/.breadcrumb/myapp-traces.duckdb",
    include=["myapp.*", "requests.*"]
)

# Your code here
def my_function():
    pass

But 99% of use cases should use breadcrumb run instead.

Environment Variables

# Override config settings
export BREADCRUMB_ENABLED=true
export BREADCRUMB_DB_PATH="/custom/path/traces.duckdb"
export BREADCRUMB_SAMPLE_RATE=0.5

---

Database Schema

Breadcrumb stores traces in DuckDB with these tables:

traces - One row per execution:

CREATE TABLE traces (
    id VARCHAR PRIMARY KEY,
    started_at TIMESTAMP,
    ended_at TIMESTAMP,
    status VARCHAR,  -- 'running', 'completed', 'failed'
    duration_ms DOUBLE
);

trace_events - Function calls and returns:

CREATE TABLE trace_events (
    id VARCHAR PRIMARY KEY,
    trace_id VARCHAR,
    timestamp TIMESTAMP,
    event_type VARCHAR,  -- 'call', 'return', 'exception', 'call_site'
    function_name VARCHAR,
    module_name VARCHAR,
    data JSON  -- Contains: args, kwargs, return_value, caller info
);

Direct SQL access:

import duckdb
conn = duckdb.connect('~/.breadcrumb/myapp-traces.duckdb')
result = conn.execute("SELECT * FROM traces").fetchall()

---

How It Works

Architecture

1. PEP 669 Backend (Python 3.12+)

   • Uses sys.monitoring API (low overhead)
   • Captures function calls/returns automatically
   • Detects untraced calls for gap detection

2. Trace Storage (DuckDB)

   • Fast columnar database
   • JSON support for flexible data
   • SQL queryable for complex analysis

3. Smart Query Layer

   • Abstracts SQL complexity
   • Returns structured JSON
   • Optimized for AI agent consumption

4. CLI Interface

   • Typer-based commands
   • Explicit project specification
   • AI-FIRST design principles

Include-Only Workflow

Default behavior: Trace only __main__ module (minimal overhead)

Iterative expansion:

breadcrumb run -c myapp python script.py     # Trace __main__ only
breadcrumb query -c myapp --gaps             # Find what's not traced
breadcrumb config edit myapp --add-include 'requests.*'  # Expand
breadcrumb run -c myapp python script.py     # Trace more

Philosophy: Start minimal, expand based on what you discover.

This is much better than tracing everything and filtering later.

---

Performance

Breadcrumb is designed for production use:

Overhead benchmarks (1M function calls):

• Baseline: 0.85s
• With Breadcrumb (PEP 669): 0.87s (2.4% overhead)
• With Breadcrumb (settrace): 0.89s (4.7% overhead)

Optimization strategies:

• Selective tracing (include patterns)
• Async database writes
• Value truncation (prevent bloat)
• Smart sampling (optional)

---

Security

Automatic secret redaction prevents sensitive data from reaching traces:

Detected patterns:

• Password fields (password, passwd, pwd, secret)
• API keys (api_key, apikey, token, bearer)
• Credit cards (16-digit patterns)
• SSNs (XXX-XX-XXXX format)
• JWTs (eyJ... tokens)
• AWS keys (AKIA... patterns)
• GitHub tokens (ghp_... patterns)

Custom patterns:

from breadcrumb.capture.secret_redactor import configure_redactor
configure_redactor(custom_patterns=['my_secret_*', 'internal_token'])

---

Requirements

• Python 3.12+ (recommended for PEP 669 backend)
• Python 3.10+ (falls back to sys.settrace backend)
• DuckDB 1.4.1+
• FastMCP 2.12.4+ (for MCP server)
• Typer 0.19.2+ (for CLI)

---

Status

Current version: 0.2.0 (Beta)

Production-ready features:

• ✅ PEP 669 tracing backend
• ✅ Zero-code-change execution (breadcrumb run)
• ✅ Gap detection (--gaps command)
• ✅ Smart queries (no SQL needed)
• ✅ AI-FIRST design (explicit -c everywhere)
• ✅ Automatic secret redaction
• ✅ DuckDB storage
• ✅ MCP server integration

Coming soon:

• 🚧 --call command (show function I/O details)
• 🚧 --flow command (execution timeline)
• 📦 PyPI package
• 📊 Performance visualizations
• ☁️ Cloud storage backends

---

Contributing

We welcome contributions! This project is built for AI coding agents.

Development setup:

git clone https://github.com/AndreRatzenberger/breadcrumb-tracer.git
cd breadcrumb-tracer
uv pip install -e .
pytest tests/

Key principles:

• AI-FIRST design (explicit over implicit)
• Zero code changes required
• Performance matters (< 5% overhead)
• Test-driven development

---

License

MIT License - see LICENSE file for details.

---

Acknowledgments

Built with:

• PEP 669 (sys.monitoring) - Low-overhead monitoring API
• DuckDB - Embedded analytical database
• FastMCP - MCP server framework
• Typer - CLI framework

Special thanks to the Model Context Protocol team at Anthropic for enabling AI-native tooling.

---

The Bottom Line

Humans debug with debuggers. AI agents debug with execution traces.

Breadcrumb gives AI coding agents the X-ray vision they need to understand what your code actually does - not just what it's supposed to do.

No code changes. No manual logging. No guessing.

Just structured execution data, queryable in real-time.

That's Breadcrumb. 🍞