pytest-flake-detection 0.1.2

Continuously runs your tests to detect flaky tests

pytest-flake-detection

Continuously runs your tests to detect flaky tests. Will default to a maximum of 99 runs, but can be configured to run indefinitely, or any number you desire.

---

This pytest plugin was generated with Cookiecutter along with @hackebrot's cookiecutter-pytest-plugin template.

Features

• This is extremely crude as of right now, but it will run your tests a number of times and report any tests which intermittently fail, along with stats on the failures.

• TODO
  • Lax mode: if a test passes continuously for the first N runs, it will be considered a pass and not run again. This is useful for tests which are slow to run, but are not flaky.
  • Randomised order: tests are run in a random order each time, to help detect flaky tests which are dependent on the order of other tests.
  • Playing nice with other plugins: this plugin will not interfere with other plugins which modify the test order, as it will randomise the order after all other plugins have had a chance to modify it.

Requirements

You will need pytest, and some electricity in your computer.

Installation

You can install "pytest-flake-detection" via pip from PyPI:

$ pip install pytest-flake-detection

Usage

$ pytest-flake-detect --max-runs 30 .

If you encounter any problems, please file an issue along with a detailed description.

Cookiecutter
@hackebrot
MIT
BSD-3
GNU GPL v3.0
Apache Software License 2.0
cookiecutter-pytest-plugin
pytest
tox
pip
PyPI