pytest plugin to re-run tests to eliminate flaky failures
pytest-rerunfailures is a plugin for py.test that re-runs tests to eliminate intermittent failures.
You will need the following prerequisites in order to use pytest-rerunfailures:
To install pytest-rerunfailures:
$ pip install pytest-rerunfailures
To re-run all test failures, use the --reruns command line option with the maximum number of times you’d like the tests to run:
$ py.test --rerun 5
To mark individual tests as flaky, and have them automatically re-run when they fail, add the flaky mark with the maximum number of times you’d like the test to run:
@pytest.mark.flaky(reruns=5) def test_example(): import random assert random.choice([True, False])
Note that when teardown fails, two reports are generated for the case, one for the test case and the other for the teardown error.
Here’s an example of the output provided by the plugin when run with --reruns 2 and -r aR:
test_report.py RRF ================================== FAILURES ================================== __________________________________ test_fail _________________________________ def test_fail(): > assert False E assert False test_report.py:9: AssertionError ============================ rerun test summary info ========================= RERUN test_report.py::test_fail RERUN test_report.py::test_fail ============================ short test summary info ========================= FAIL test_report.py::test_fail ======================= 1 failed, 2 rerun in 0.02 seconds ====================
Note that output will show all re-runs. Tests that fail on all the re-runs will be marked as failed. Due to a current limitation in pytest-xdist, when running tests in parallel only the final result will be included in the output.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|File Name & Checksum SHA256 Checksum Help||Version||File Type||Upload Date|
|pytest_rerunfailures-2.2-py2.py3-none-any.whl (8.5 kB) Copy SHA256 Checksum SHA256||py2.py3||Wheel||Jun 23, 2017|
|pytest-rerunfailures-2.2.tar.gz (7.9 kB) Copy SHA256 Checksum SHA256||–||Source||Jun 23, 2017|