bottrace 0.1.0

A headless CLI debugging controller for Python

bottrace

A headless CLI debugging controller for Python. Designed for both humans and LLM agents.

Installation

pip install -e .

Quick Start

# Basic call tracing
bottrace run app.py --calls

# See what functions are called most
bottrace run app.py --call-counts

# Debug an exception
bottrace run buggy.py --on-exception snapshot --locals

# Breakpoint snapshots (capture state at specific lines)
bottrace run app.py --breakpoint app/core.py:42 --watch 'len(items)' --json-out trace.json

# Bounded execution (prevent runaway traces)
bottrace run app.py --calls --timeout 10 --max-events 1000

Core Concept

bottrace is not a debugger. It's a debugging controller.

It wraps Python's sys.settrace() and related primitives to emit structured, machine-parseable output. This lets you understand runtime behavior without setting breakpoints or adding print statements.

Key principle: Runtime execution is authoritative. Symbol names and static analysis are secondary to observed call paths.

Commands

bottrace run

Execute a Python script with optional tracing and debugging controls.

bottrace run <script.py> [script_args...] [options]

Options

Tracing Options

Flag	Description
--calls	Emit CALL/RETURN events for each function
--call-counts	Show aggregated call frequency summary (Note: Misses C → Python calls)
--no-comprehensions	Suppress comprehension/genexpr call events
--max-depth N	Limit trace depth to N levels

Filtering Options

Flag	Description
--include PREFIX	Only trace files under this path (repeatable)
--exclude PREFIX	Exclude files under this path (repeatable)

By default, bottrace traces only your project code (script's directory), not site-packages or stdlib.

Exception Options

Flag	Description
--on-exception snapshot	On uncaught exception, dump stack trace
--locals	Include local variables in exception snapshot

Bounding Options

Flag	Description
--timeout N	Kill execution after N seconds (exit code 124)
--max-events N	Stop tracing after N events (program continues)

Breakpoint Snapshot Options

Flag	Description
--breakpoint FILE:LINE	Capture snapshot when execution reaches file:line (repeatable)
--watch EXPR	Expression to evaluate at breakpoints (repeatable)
--snapshots N	Stop capturing after N snapshots
--json-out PATH	Write snapshots to JSON file
--verbose	Print snapshot summaries to stderr (useful without JSON)
--print-result	Print JSON result summary to stdout (for agent discovery with --json-out)

Breakpoint snapshots are non-interactive - execution auto-continues after each capture. Designed for CI/CD and LLM agent workflows.

Important: Breakpoint Placement

Set breakpoints on lines inside function bodies, not on def lines:

def process(item):      # Line 10 - DON'T set breakpoint here (captures at import time)
    result = item * 2   # Line 11 - SET breakpoint here (captures at call time)
    return result       # Line 12 - Or here

The def line executes when the module is imported (to create the function object), not when the function is called. To capture state during function execution, use a line inside the function body.

Output Format

Call Trace

CALL main app.py:10
CALL load_config app.py:5
RETURN load_config
CALL process app.py:15
RETURN process
RETURN main

Call Counts

Call Counts (Top 10):
 100 calls to inner (worker.py:12)
   1 calls to main (app.py:5)
   1 calls to outer (app.py:10)

Exception Snapshot

Exception: KeyError(user_id)

Stack:
  get_user() db.py:42
    Locals:
      user_id: 'abc123'
      conn: <Connection connected=True>
  handle_request() api.py:28
  main() app.py:10

Timeout

TIMEOUT after 5s - dumping stacks:
Thread MainThread:
  infinite_loop() slow.py:7
  main() slow.py:2

Breakpoint Snapshot (JSON)

{
  "version": 1,
  "snapshots": [
    {
      "breakpoint": "app/core.py:42",
      "timestamp": 1706640000.123,
      "stack": [
        {
          "function": "process_item",
          "filename": "/home/user/app/core.py",
          "lineno": 42,
          "locals": {"item": "{'id': 123}", "count": "5"}
        },
        {
          "function": "main",
          "filename": "/home/user/app/main.py",
          "lineno": 25,
          "locals": null
        }
      ],
      "watches": [
        {"expression": "len(items)", "value": "42", "error": null},
        {"expression": "self.state", "value": "'running'", "error": null}
      ]
    }
  ]
}

Exit Codes

Code	Meaning
0	Success
1-123	Script's exit code (passed through)
124	Timeout

Usage Patterns for Agents

1. Find all code executed by a feature

# Trace the feature
bottrace run manage.py migrate --calls > trace.txt

# Extract unique files touched
grep "^CALL" trace.txt | awk '{print $3}' | cut -d: -f1 | sort -u

# Get call frequency
bottrace run manage.py migrate --call-counts

2. Disambiguate same-named symbols

When multiple modules define FooHandler, prove which one runs:

bottrace run app.py handle_request --calls | grep FooHandler
# Output: CALL FooHandler handlers/v2/foo.py:42

3. Verify no forbidden dependencies

Confirm NewClass doesn't call into legacy code:

bottrace run test_new_class.py --calls > trace.txt
grep "legacy_module" trace.txt
# Empty = pass, any output = violation

4. Debug without adding logging

# Use existing logging + bottrace together
LOG_LEVEL=DEBUG bottrace run app.py --calls --max-depth 5 2>&1 | tee output.txt

5. Find why something is called 10,000 times

bottrace run app.py --call-counts
# Shows: 10000 calls to validate_item (validator.py:12)

6. Understand end-to-end flow

bottrace run cli.py deploy --calls --max-depth 4
# Shows: boot -> dispatch -> business logic -> side effect

7. Debug an exception

bottrace run buggy.py --on-exception snapshot --locals
# Shows stack + local variables at crash point

8. Capture state at specific locations (breakpoint snapshots)

# Capture stack + locals when hitting specific lines
bottrace run app.py \
  --breakpoint app/core.py:42 \
  --breakpoint app/api.py:88 \
  --watch 'len(items)' \
  --watch 'request.user_id' \
  --snapshots 5 \
  --json-out debug.json

# Then analyze the JSON
jq '.snapshots[0].stack[0].locals' debug.json
jq '.snapshots[] | .watches[] | select(.error == null)' debug.json

This is ideal for:

• Capturing state in CI/CD without interactive debugging
• LLM agents that need structured debug data
• Repeatable "debug snapshots" for regression analysis

LLM Agent Workflow

When using bottrace as an LLM agent, follow this 3-step process:

Step 1: Get the Shape

Start with --call-counts to understand what's happening without drowning in detail:

bottrace run app.py --call-counts --timeout 5

Output shows the most-called functions with exact file:line locations:

Call Counts (Top 10):
 892 calls to __init__ (unified_atlas_manager.py:144)
 101 calls to _ (i18n.py:133)
  88 calls to __init__ (sound_manager.py:41)

Step 2: Map Files Touched

Extract unique project files executed:

bottrace run app.py --calls --max-events 5000 2>&1 | \
  grep "^CALL" | awk '{print $3}' | cut -d: -f1 | sort -u

This gives you the complete list of files involved in the execution path.

Step 3: Drill Into Specific Areas

Focus on a specific module or limit depth:

# Focus on one module
bottrace run app.py --calls --include src/auth/ --max-events 1000

# Or limit call depth for high-level view
bottrace run app.py --calls --max-depth 3 --max-events 1000

Real-World Example: Tracing a Pygame Game

Tested on a 199-file Pygame/OpenGL game:

# Run headlessly with virtual framebuffer
SDL_AUDIODRIVER=dummy xvfb-run -a \
  bottrace run src/main.py --windowed --call-counts --timeout 5

Finding Hotspots

bottrace run src/main.py --calls --max-events 50000 2>&1 | \
  grep "^CALL" | awk '{print $2, $3}' | sort | uniq -c | sort -rn | head -10

Output:

10091 <genexpr> zaxxon_turret.py:766
 3127 <genexpr> zaxxon_turret.py:814
 1541 normalize_vec3 math_helpers.py:456
  786 add_vec3 math_helpers.py:475

This immediately reveals that zaxxon_turret.py has hot generator expressions worth investigating.

Symbol Disambiguation

When multiple modules define the same function (e.g., normalize):

grep -rn "def normalize" src/
# src/math_helpers.py:456:def normalize_vec3(v)
# src/behaviors/super_formation_join_behavior.py:361:def normalize(angle)
# src/bullet.py:109:def normalize(self, vector)

bottrace proves which one actually runs:

1541 calls to normalize_vec3 (math_helpers.py:456)

Decoupling Verification

Verify that module A doesn't call into forbidden module B:

# Capture full trace
bottrace run app.py --calls --max-events 50000 > trace.txt

# Check: does camera.py call adversary code?
grep "camera.py" trace.txt | grep -c "adversary"
# Output: 0 (PASS - decoupled)

# Check: does i18n call rendering code?
grep "i18n.py" trace.txt | grep -c "renderer"
# Output: 0 (PASS - decoupled)

High-Level Flow with --max-depth

bottrace run src/main.py --calls --max-depth 2 --max-events 500

Output shows boot sequence without internal details:

CALL <module> src/main.py:1
CALL <module> src/TwoD.py:1
RETURN <module>
CALL <module> src/game_state.py:1
RETURN <module>
...
CALL main src/main.py:1447
RETURN main

Progressive Narrowing Workflow

bottrace is designed for iterative refinement:

1. Broad trace - bottrace run app.py --calls
2. Identify noise - Note irrelevant modules
3. Narrow - bottrace run app.py --calls --exclude tests/ --max-depth 3
4. Repeat - Until you see only relevant code

Combining Flags

Flags can be placed before or after the script path:

# Both work:
bottrace run --calls --max-depth 3 app.py
bottrace run app.py --calls --max-depth 3

# Script args come last:
bottrace run app.py --calls -- --script-arg value

Examples

The examples/ directory contains test scripts:

• hello.py - Basic script with args
• calls.py - Nested function calls
• exception.py - Script that raises
• slow.py - Long-running script (for timeout testing)
• many_calls.py - High call volume (for call-counts testing)

Limitations

• C-Extension Boundary: sys.settrace() does not fire CALL events when a C-extension calls a Python function (e.g. pygame main loop, numpy callbacks). These calls are missed by --calls and --call-counts. Workaround: use --breakpoint on the first line of the Python function.
• No attach: Cannot attach to running processes
• No continuous line tracing: Only call/return events by default (use --breakpoint for specific lines)
• No async tracing: Does not trace async task switches
• Watch expressions: Limited to safe builtins (no imports, no side effects)

Design Philosophy

1. CLI-first - No UI, works over SSH/CI/containers
2. Unix composable - Pipe to grep, jq, awk
3. Agent-friendly - Deterministic output, explicit flags
4. Bounded by default - Always use caps to prevent drowning
5. Execution is truth - Runtime traces beat static analysis

stdout/stderr Contract

Golden rule: STDOUT is data. STDERR is commentary.

This enables clean Unix piping and agent-friendly parsing.

What Goes Where

Output Type	Stream	Example
CALL/RETURN trace events	stdout	CALL main app.py:10
Call counts summary	stdout	100 calls to inner (worker.py:12)
NDJSON snapshots (no --json-out)	stdout	{"breakpoint":...}
Exception snapshots	stdout	Exception: KeyError(...)
--print-result JSON	stdout	{"result_file":"...","snapshots":3}
Errors and warnings	stderr	bottrace: error: script not found
--verbose summaries	stderr	SNAPSHOT #1 at file:42
MAX_EVENTS/MAX_SNAPSHOTS notices	stderr	MAX_EVENTS reached (5)
Timeout messages	stderr	TIMEOUT after 5s - dumping stacks

Agent Discovery with --print-result

When writing to a JSON file, use --print-result for deterministic artifact discovery:

bottrace run app.py --breakpoint app.py:42 --json-out trace.json --print-result
# STDOUT: {"result_file": "trace.json", "snapshots": 3}

This lets agents capture the result without parsing stderr:

RESULT=$(bottrace run ... --json-out trace.json --print-result)
FILE=$(echo "$RESULT" | jq -r '.result_file')
COUNT=$(echo "$RESULT" | jq -r '.snapshots')

For Agent Developers

When training an agent to use bottrace:

1. Start with --call-counts - Get the shape before drilling down
2. Always bound output with --max-events (5000-50000) or --timeout
3. Use --max-depth 2-3 for architecture understanding
4. Extract file lists with grep/awk pipeline for codebase mapping
5. Verify decoupling by grepping traces for forbidden modules
6. Cite evidence - conclusions should reference bottrace output with file:line
7. Don't add logging - use --on-exception snapshot --locals instead

Output Parsing Tips

The output format is designed for Unix pipelines:

# Count calls per file
grep "^CALL" trace.txt | awk '{print $3}' | cut -d: -f1 | sort | uniq -c | sort -rn

# Find all calls to a specific function
grep "^CALL specific_func" trace.txt

# Check if module X calls module Y
grep "module_x" trace.txt | grep -c "module_y"  # 0 = no coupling

Recommended Bounds

Scenario	Recommended Flags
Quick overview	--call-counts --timeout 5
File mapping	--calls --max-events 5000
Architecture	--calls --max-depth 3 --max-events 1000
Focused debugging	--calls --include src/module/ --max-events 2000
Exception analysis	--on-exception snapshot --locals
Breakpoint debugging	--breakpoint file:line --snapshots 5 --json-out trace.json
Quick breakpoint check	--breakpoint file:line --verbose (no JSON, summaries to stderr)

See docs/USE_CASES.md for detailed agent workflow patterns.