OracleTrace — Detect Python Performance Regressions with Execution Diff

Detect Python performance regressions before they merge.

Fail your CI when performance regresses. OracleTrace is a git diff for performance. Create a known-good baseline, trace the current run, and see exactly which functions got slower.

Documentation: https://kaykcaputo.github.io/oracletrace/

Featured in: PyCoder's Weekly #729 • La Experimental #30 • Python技术周刊 #15,#16 and #17 • Woudar's Blog #214 • awesome-debugger • awesome-profiling

Installation

pip install oracletrace

Quick Start

1. See where your program spends time instantly:

oracletrace app.py

2. Compare runs and detect regressions:

oracletrace app.py --json baseline.json
oracletrace app.py --json new.json --compare baseline.json

3. Manage baselines explicitly:

oracletrace baseline save app.py baseline.json
oracletrace app.py --json current.json
oracletrace baseline compare baseline.json current.json --fail-on-regression --threshold 10

Examples

Try these ready-to-run scripts to explore OracleTrace features:

# CPU hotspot — highlights the heaviest function
oracletrace examples/cpu_hotspot.py

# Nested call graph — see the tree visualization
oracletrace examples/nested_calls.py

# Regression demo — baseline vs slower path
oracletrace examples/regression_demo.py --json baseline.json
SLOW=1 oracletrace examples/regression_demo.py --json current.json --compare baseline.json

All examples are deterministic, finish in under a second, and live in examples/. They can also be reused in test suites as smoke tests.

See it in action

See exactly which functions got slower between runs:

Example Output

Starting application...

Iteration 1:
  > Processing data...
    > Calculating results...

Iteration 2:
  > Processing data...
    > Calculating results...

Application finished.

Summary:
                         Top functions by Total Time
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ Function                     ┃ Total Time (s) ┃ Calls ┃ Avg. Time/Call (ms) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ my_app.py:main               │         0.6025 │     1 │             602.510 │
│ my_app.py:process_data       │         0.6021 │     2 │             301.050 │
│ my_app.py:calculate_results  │         0.4015 │     2 │             200.750 │
└──────────────────────────────┴────────────────┴───────┴─────────────────────┘

Logic Flow:
<module>
└── my_app.py:main (1x, 0.6025s)
    └── my_app.py:process_data (2x, 0.6021s)
        └── my_app.py:calculate_results (2x, 0.4015s)

Why OracleTrace?

Problem

Performance regressions often reach production because correctness tests still pass.

Solution

OracleTrace gives CI a small, scriptable performance gate based on saved execution traces.

How it works

1. Run your script
2. Generate a trace
3. Compare results
4. Identify slowdowns

CI Integration

Fail your pipeline when performance degrades:

oracletrace run \
  --repeat 50 \
  --json current.json \
  --compare baseline.json \
  --fail-on-regression \
  --threshold 35 \
  -- pytest tests/

Add it to your CI to automatically fail on performance regressions.

Baseline-file workflow:

# on a stable branch or release job
oracletrace baseline save app.py baseline.json --repeat 50

# in a pull request job
oracletrace app.py --json current.json --repeat 50
oracletrace baseline compare baseline.json current.json --fail-on-regression --threshold 35

Key Features

• Detect slower and faster functions
• Identify new or removed functions
• Execution time and call count analysis
• Call graph visualization
• JSON and CSV export
• Regex-based filtering (--ignore)
• Top-N function focus (--top)
• CI regression gates
• Baseline save and compare commands

CLI Reference

Baseline commands:

oracletrace baseline save app.py baseline.json
oracletrace baseline compare baseline.json current.json --fail-on-regression --threshold 35

Use Cases

Primary

• Guard pull requests against Python performance regressions

Secondary

• CI performance validation
• Execution trace inspection
• Call graph visualization
• Release baseline tracking

How It Works

OracleTrace uses Python’s sys.setprofile() to intercept function calls and returns.

It measures execution time per function and records caller–callee relationships.

Filtering removes external/internal calls to focus on application code.

Requirements

• Python >= 3.11
• rich

When to use another tool

OracleTrace is for lightweight regression checks and function-level change detection. Use cProfile, py-spy, or a benchmark suite when you need deep profiler statistics, sampling of production processes, memory profiling, or statistically rigorous microbenchmarks.

Contributing

Contributions are welcome.

Please read the Contributing Guide for details on how to get started, coding standards, and the contribution process.

Contributors

⭐ Support the Project

If OracleTrace is useful, consider giving it a star:

GitHub Repository

Maintainers

• Kayk Caputo
• André Gustavo