Skip to main content

Saves shell scripts that allow re-execute previous pytest runs to reproduce crashes or flaky tests

Project description

See Build Status on Travis CI See Build Status on AppVeyor

Saves shell scripts that allow re-execute previous pytest runs to reproduce crashes or flaky tests


This Pytest plugin was generated with Cookiecutter along with @hackebrot’s Cookiecutter-pytest-plugin template.

Features

This plugin helps to reproduce random or flaky behavior when running tests with xdist: because xdist executes order in a non-predictable manner, sometimes it is hard to reproduce a behavior seen in CI locally.

This plugin introduces a new --replay-script-dir flag where shell scripts will be recorded containing a footprint of the tests executed by each worker during the session. For example:

$ pytest -n auto --replay-script-dir build/tests/replay

Will now generate platform specific shell-scripts (.bat for Windows and .sh for Linux) in the build/tests/replay directory which record each test executed by each worker, in a ready-to-use command-line to reproduce it later. For example, it will generate a shell script for worker gw1 named .pytest-replay-gw1.sh with this contents:

# generated by pytest-replay
pytest $* \
  test_foo.py::test[1] \
  test_foo.py::test[3] \
  test_foo.py::test[5] \
  test_foo.py::test[7] \
  test_foo.py::test[8] \

So to (hopefully) reproduce a flaky behavior seen in CI one can execute that script locally to run the tests in the same order.

Installation

You can install pytest-replay via pip from PyPI:

$ pip install pytest-replay

Or with conda:

$ conda install -c conda-forge pytest-replay

Contributing

Contributions are very welcome.

Tests can be run with tox if you are using a native Python installation.

To run tests with conda, first create a virtual environment and execute tests from there (conda with Python 3.5+ in the root environment):

$ python -m venv .env
$ .env\scripts\activate
$ pip install -e . pytest-xdist
$ pytest tests

Releases

Follow these steps to make a new release:

  1. Create a new branch release-X.Y.Z from master;

  2. Update CHANGELOG.rst;

  3. Open a PR;

  4. After it is green and approved, push a new tag in the format X.Y.Z;

Travis will deploy to PyPI automatically.

Afterwards, update the recipe in conda-forge/pytest-replay-feedstock.

License

Distributed under the terms of the MIT license.

Issues

If you encounter any problems, please file an issue along with a detailed description.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest-replay-0.1.1.tar.gz (8.1 kB view hashes)

Uploaded Source

Built Distribution

pytest_replay-0.1.1-py3-none-any.whl (6.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page