Coverage flashlight

See which Python code ran, then replay its execution as a flashlight revealing source through fog. Reports are standalone HTML with source snapshots, shared light/dark themes, and no server or external assets.

Install and run

Install in the environment containing the program and its dependencies:

uv add --dev 'coverage-flashlight[pytest]'
# Or: python -m pip install 'coverage-flashlight[pytest]'

Usage

# Static per-test line/branch coverage; pytest arguments follow --.
flashlight pytest -- tests/ -k database

# Ordered playback, including threads and newly launched Python processes.
flashlight pytest --trace -- tests/ -k database

# A script, with its arguments, executed once.
flashlight run -- scripts/example.py --input sample.bin
flashlight run --trace -- scripts/example.py --input sample.bin

# Python modules work too, including other test runners.
flashlight run -m my_package --help
flashlight run --trace -m unittest discover

# Render existing coverage.py branch data without another run.
flashlight report

Open htmlcov/flashlight.html for static coverage or htmlcov/execution.html for playback. Use --output path/to/report.html before -- to choose another destination. python -m coverage_flashlight is equivalent to flashlight.

Coverage and tracing are separate modes: a command never silently runs a script twice. --trace selects execution recording instead of coverage.py. To create both reports, run the two commands explicitly. The program's exit status is preserved and failed runs still produce a report. Pytest mode continues recording other cases after a failure and returns a nonzero status.

Pytest is an optional adapter. Static measurement uses coverage.py; ordered playback uses Python trace events. Script/module execution requires no pytest installation. Each pytest case runs in its own interpreter with its own context, which also follows newly launched Python children. This includes fixture setup and teardown. Isolation is slower than a normal suite run and can expose fixture or order dependencies; continue running ordinary tests for correctness.

Source selection

Run from the target project's root. By default, the tool includes Python files inside that root, excluding hidden directories, virtual environments, build outputs, and node_modules. Narrow this with --source src --source tests or:

[tool.coverage-flashlight]
source = ["src/my_package"]
trace-source = ["src/my_package", "tests"]
omit = ["*/generated/*"]

Paths are project-relative files or directories, not import names. trace-source defaults to source; absent both, the project root is used. --source overrides the configured selection for the current command. Keep source files inside the project root. Selected but unexecuted files remain visible in the reports.

Coverage mode writes fresh data to .coverage after collection, replacing its previous contents. Use --data-file path to keep a separate database. It enables branch measurement and subprocess/os._exit() coverage itself; no coverage.py configuration is required. Existing report exclusion rules are respected when rendering. flashlight report can also render a combined database collected by coverage.py or pytest-cov, preserving its context names.

Raw recordings and manifests are kept beside the report under runs/<run-id>/. The default trace cap is 100,000 events per process; increase it with --max-events 500000. Capture caps, interrupted hooks, source I/O failures, and missing process completion markers are explicitly reported.

Reading the reports

Static coverage distinguishes executed lines, branch gaps, execution in another scope, and unexecuted statements. Select a line to inspect its contexts and missing branch destinations. Coverage is evidence of execution, not proof of assertions.

Playback offers Play/Pause, stepping, timeline scrubbing, Next new line, a pytest Test body shortcut, Whole run, and process/thread selection. Unrevealed ranges collapse to grey … rows; Reveal source context expands them without marking them visited. Selecting a module pauses playback; Follow execution restores automatic navigation. The module map and source scroll inside the window.

The timeline records observed event order, not variable history or cross-thread causality. Speed is events per second, not original elapsed time. Generator suspension produces Python return/call events. Native code, generated code with synthetic filenames, pre-existing processes, and threads created outside Python's threading module are outside capture. A forcibly killed process can lose its buffered tail. A skipped test can have no events. The source snapshot must remain consistent across cases; changed source is rejected rather than misattributed.

Tracing changes timing and cannot share a trace hook with coverage.py or a debugger. Pytest-xdist parallel execution is rejected: per-case subprocess isolation already supplies attribution. Python children using -S or -I can bypass the opt-in bootstrap. No startup hook is installed globally.

Development and releases

uv sync --all-extras
uv run pytest
uv run ruff check .
uv run ruff format --check .
uv build

Tests use disposable Python projects, real threads/subprocesses, failing and skipped cases, arbitrary pytest layouts, and actual script/module entry points. They require no IDA installation. HTML, CSS and the opt-in bootstrap ship in the wheel and source distribution.

The manual Release GitHub Actions workflow supports release-current, dev, release-patch, and release-minor. It tests, builds and checks the wheel, commits an optional version bump, creates a tag, publishes with uv publish, and creates a GitHub release. Versioning uses uv version; there is only one package version.

To enable publication, create the GitHub repository and configure a PyPI trusted publisher for project coverage-flashlight, your repository owner/name, workflow release.yml, and environment pypi. Create that GitHub environment and allow the workflow to push release commits/tags. No PyPI API token is needed. These setup steps do not publish anything until the workflow is dispatched.

References: uv publishing, coverage.py subprocess measurement, Python tracing.