flakeradar 🎯

Flaky-test radar for pytest. Tracks per-test flake rate across runs and writes a FLAKY.md leaderboard your CI commits back — so flaky tests are named, ranked, and impossible to ignore.

  • Zero dependencies beyond pytest. Pure stdlib.
  • History is a plain JSONL file (.flakeradar/history.jsonl) you commit to the repo — that's what makes "across runs" work on stateless CI runners.
  • Scores flakiness, not failure: a test that always fails scores 0 (it's broken, not flaky). A test that alternates pass/fail scores near 1.

Install

pip install pytest-flakeradar

Use

pytest --flakeradar

Every run appends one record to .flakeradar/history.jsonl, rescores the window, and rewrites FLAKY.md:

# Test Flake score Recoveries Flips Fail / Pass Last outcome
1 tests/test_api.py::test_ws_reconnect 0.75 1 2 2 / 2 passed

Flake score = (intra-run recoveries + inter-run flips) / runs seen, over the last --flakeradar-window runs (default 50).

  • Recovery: failed then passed within one run (e.g. pytest-rerunfailures retries). The strongest flake evidence there is.
  • Flip: final outcome changed between consecutive runs.

Options

Flag Default
--flakeradar off enable recording + leaderboard
--flakeradar-dir .flakeradar history directory (relative to rootdir)
--flakeradar-md FLAKY.md leaderboard path; "" disables
--flakeradar-window 50 runs to score over
--flakeradar-keep 200 max runs kept in history
--flakeradar-top 20 max leaderboard rows
--flakeradar-fail-above SCORE off exit non-zero if any score ≥ SCORE (CI gate)

CLI (no pytest run needed)

flakeradar report            # print the leaderboard
flakeradar md --out FLAKY.md # regenerate the markdown

CI setup (the commit-back loop)

The runner is stateless, so persist history by committing it back. GitHub Actions: see examples/github-actions.yml — run pytest with --flakeradar, then commit .flakeradar/history.jsonl + FLAKY.md with [skip ci]. GitLab and Buildkite equivalents live in examples/ too.

Prefer not to commit? Restore/save .flakeradar/ with your CI cache instead — the leaderboard still works, it's just not visible in the repo.

CI providers

flakeradar stamps each run with provider metadata (run id/URL/branch/commit), detected from environment variables. Built-in adapters: GitHub Actions, GitLab CI, Buildkite, CircleCI, Jenkins, with a local fallback.

Your provider missing? That's a ~20-line PR. See CONTRIBUTING.md — one small class with detect(env) and run_meta(env). Third-party packages can also register adapters via the flakeradar.adapters entry-point group without touching this repo.

License

MIT © Sophie Nguyen (Nguyễn Thu Thuỷ)

Project links

Data verified by PyPI on Aug 9, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for sophie.nguyenthuthuy from gravatar.com sophie.nguyenthuthuy

Credits

Author: Sophie Nguyen (Nguyễn Thu Thuỷ)

GitHub Statistics

Data verified by PyPI on Aug 9, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.9

Tags

ci flaky flaky-tests pytest test-reliability

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest_flakeradar-0.1.0.tar.gz (13.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pytest_flakeradar-0.1.0-py3-none-any.whl (12.6 kB view details)

Uploaded Python 3

File details

Details for the file pytest_flakeradar-0.1.0.tar.gz.

File metadata

  • Download URL: pytest_flakeradar-0.1.0.tar.gz
  • Upload date:
  • Size: 13.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for pytest_flakeradar-0.1.0.tar.gz
Algorithm Hash digest
SHA256 32c9b5c168b4b384838f149c123fc5edb07a6318b2757f8bb4e2e8d8f5e8e297
MD5 130941fec896db29b77ec04d1daabc10
BLAKE2b-256 e90484bac281de0cba35884f9ea344bccfc425ee901cbc6e9d2f123d113f5a86

See more details on using hashes here.

Provenance

The following attestation bundles were made for pytest_flakeradar-0.1.0.tar.gz:

Publisher: release.yml on sophie-nguyenthuthuy/flakeradar

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pytest_flakeradar-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for pytest_flakeradar-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 95036488977c2e0e6affbe3db6f2f53b8f281cee22b23e0848251ab8fa2f2e64
MD5 bbd6258b5350c8214b34fc187dab6e42
BLAKE2b-256 ee4dc897fa6ee76cbcc0361b3f497406e520dc1c1f6b146c8974c2037c5d5fb1

See more details on using hashes here.

Provenance

The following attestation bundles were made for pytest_flakeradar-0.1.0-py3-none-any.whl:

Publisher: release.yml on sophie-nguyenthuthuy/flakeradar

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page