Skip to main content

Find tests leaking state and affecting other

Project description

pytest-bisect-tests

Sometimes one test can affect the execution of another test. This simple script tries to find the offender.

This script can be called as standalone script, or via pytest option.

Example:

# faulty_test.py
a = 1

def test_faulty() -> None:
    global a
    a = 2

def test_failing() -> None:
    assert a == 1


$ pytest-bisect-tests --failing-test "faulty_test.py::test_failing"
Checking if the test fails on its own.
Running 1 tests.
Trying to find the faulty test.
Running 2 tests.
Faulty test: faulty_test.py::test_faulty

Comparison of UX

Standalone script: asciicast

pytest option: asciicast

Installation

$ pip install pytest-bisect-tests

Usage

Standalone script

$ pytest-bisect-tests --failing-test "<identifier of the test as pytest shows them with -v>"

Additional arguments:

$ pytest-bisect-tests --help
usage: pytest-bisect-tests [-h] --failing-test FAILING_TEST [--run-options RUN_OPTIONS] [--collect-options COLLECT_OPTIONS] [--stdout]

options:
  -h, --help            show this help message and exit
  --failing-test FAILING_TEST
                        REQUIRED. The identifier of the test, as shown by pytest -v.
  --run-options RUN_OPTIONS
                        Arguments that will be passed directly to pytest. A single string.
  --collect-options COLLECT_OPTIONS
                        Arguments that will be passed to pytest during tests collection. A single string.
                        This is useful when, for example, you have a test grouping plugin that affects the tests run.
  --stdout              If passed, pytest output will be shown.

As pytest option

This approach automatically discovers all tests in the suite and failing test, so it has minimal input options.

$ pytest --bisect-first-failure

Alternatives

detect-test-pollution, an alternative package with similar functionality. Lacks passing pytest run and collect options or showing the output from pytest. It also doesn't support pytest plugins that alter how tests are collected (like pytest-test-groups).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest_bisect_tests-0.3.0.tar.gz (5.6 kB view hashes)

Uploaded Source

Built Distribution

pytest_bisect_tests-0.3.0-py3-none-any.whl (7.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page