Skip to main content

Find tests leaking state and affecting other

Project description

pytest-bisect-tests

Sometimes one test can affect the execution of another test. This simple script tries to find the offender.

This script can be called as standalone script, or via pytest option.

Example:

# faulty_test.py
a = 1

def test_faulty() -> None:
    global a
    a = 2

def test_failing() -> None:
    assert a == 1


$ pytest-bisect-tests --failing-test "faulty_test.py::test_failing"
Checking if the test fails on its own.
Running 1 tests.
Trying to find the faulty test.
Running 2 tests.
Faulty test: faulty_test.py::test_faulty

Comparison of UX

Standalone script: asciicast

pytest option: asciicast

Installation

$ pip install pytest-bisect-tests

Usage

Standalone script

$ pytest-bisect-tests --failing-test "<identifier of the test as pytest shows them with -v>"

Additional arguments:

$ pytest-bisect-tests --help
usage: pytest-bisect-tests [-h] --failing-test FAILING_TEST [--run-options RUN_OPTIONS] [--collect-options COLLECT_OPTIONS] [--stdout]

options:
  -h, --help            show this help message and exit
  --failing-test FAILING_TEST
                        REQUIRED. The identifier of the test, as shown by pytest -v.
  --run-options RUN_OPTIONS
                        Arguments that will be passed directly to pytest. A single string.
  --collect-options COLLECT_OPTIONS
                        Arguments that will be passed to pytest during tests collection. A single string.
                        This is useful when, for example, you have a test grouping plugin that affects the tests run.
  --stdout              If passed, pytest output will be shown.

As pytest option

This approach automatically discovers all tests in the suite and failing test, so it has minimal input options.

$ pytest --bisect-first-failure

Alternatives

detect-test-pollution, an alternative package with similar functionality. Lacks passing pytest run and collect options or showing the output from pytest. It also doesn't support pytest plugins that alter how tests are collected (like pytest-test-groups).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest_bisect_tests-0.3.0.tar.gz (5.6 kB view details)

Uploaded Source

Built Distribution

pytest_bisect_tests-0.3.0-py3-none-any.whl (7.2 kB view details)

Uploaded Python 3

File details

Details for the file pytest_bisect_tests-0.3.0.tar.gz.

File metadata

  • Download URL: pytest_bisect_tests-0.3.0.tar.gz
  • Upload date:
  • Size: 5.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for pytest_bisect_tests-0.3.0.tar.gz
Algorithm Hash digest
SHA256 9f0665a284cf3c3fdaf304bad761cd31b5bf5429a311abf9e5ce32c3135b9614
MD5 625fb88f11987c6eafba8959b32407a1
BLAKE2b-256 ec02e8211963a8c9c8e59d73d212f96dbcc564864108e86cdb41da1b94d11999

See more details on using hashes here.

File details

Details for the file pytest_bisect_tests-0.3.0-py3-none-any.whl.

File metadata

File hashes

Hashes for pytest_bisect_tests-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 53de8e174b4d6bd81bbaed0563fb1631d51b10adb2ec0b057ab89fb25a2bc731
MD5 d25f167155e194e227f41768fe954155
BLAKE2b-256 fac3d079b7634a8d5e3163331c2e6857480b270f1008696c640e710996993c1a

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page