scope-profiler

This module provides a unified profiling system for Python applications, with optional integration of LIKWID markers using the pylikwid marker API for hardware performance counters.

It allows you to:

• Configure profiling globally via a singleton ProfilingConfig.
• Collect timing data via context-managed profiling regions.
• Use a clean decorator syntax to profile functions.
• Optionally record time traces in HDF5 files.
• Automatically initialize and close LIKWID markers only when needed.
• Print aggregated summaries of all profiling regions.

Install

Install from PyPI:

pip install scope-profiler

Usage

To set up the configuration, create an instance of ProfilingConfig and add it to the ProfileManager, this should be done once at application startup and will persist until the program exits or is explicitly finalized (see below). Note that the config applies to any profiling contexts created (even in other files) after it has been initialized.

from scope_profiler import ProfileManager

# Setup global profiling configuration
ProfileManager.setup(
    use_likwid=False,
    recursive_profile=False,
    time_trace=True,
    flush_to_disk=True,
)

# Profile the main() function with a decorator
@ProfileManager.profile("main")
def main():
    x = 0
    for i in range(10):
        # Profile each iteration with a context manager
        with ProfileManager.profile_region(region_name="iteration"):
            x += 1

# Call main
main()

# Finalize profiler
ProfileManager.finalize()

Execution:

❯ python test.py
Region: main
  Total Calls : 1
  Total Time  : 0.001503709 s
  Avg Time    : 0.001503709 s
  Min Time    : 0.001503709 s
  Max Time    : 0.001503709 s
  Std Dev     : 0.0 s
----------------------------------------
Region: iteration
  Total Calls : 10
  Total Time  : 3.832e-06 s
  Avg Time    : 3.832e-07 s
  Min Time    : 2.08e-07 s
  Max Time    : 8.75e-07 s
  Std Dev     : 2.2431888016838885e-07 s
----------------------------------------

Overhead

The profiling overhead per call depends on the region type. The benchmark below (examples/benchmark_overhead.py) measures each mode against a bare function call:

The two modes most relevant to HPC — NCallsOnly and TimeOnly — add roughly 0.09 µs and 0.75 µs per instrumented call respectively.

Profiling can also be fully deactivated at setup time (profiling_activated=False) to reduce the overhead to ~0.03 µs — barely above a bare function call — making it safe to leave instrumentation in production code and toggle it on only when needed.

The LineProfiler mode is intentionally heavier (~41 µs/call) because line_profiler traces every source line. It is designed for targeted debugging of individual functions, not for always-on use in hot loops.

Recursive profiling of nested calls

You can profile nested Python calls from one decorated entrypoint:

from scope_profiler import ProfileManager

ProfileManager.setup(recursive_profile=True)


def leaf(x):
    return x + 1


def inner(x):
    return leaf(x) * 2


@ProfileManager.profile("entry")
def entry():
    return sum(inner(i) for i in range(3))


entry()
ProfileManager.finalize()

When enabled, the profiler records regions for nested calls using fully qualified names (for example, my_module.inner), in addition to the main decorated region.