Find the previous test that makes another to fail
Project description
pytest-leak-finder
You have a test that passes when executed alone but fails when running its suite. What's happening? My two cents that some previous test keeps the things dirty. But wich one/s, maybe the previous are a lot, right?
This plugin helps to find a culprit by doing a binary search (alla git bisect) on the collected tests before the target.
The first time it will collect the first half of those tests plus the failing one (the target). If the target fails, we are in a good path, so, a new bisect is applied. When the target doesn't fail, it changes the "half" to bisect the next time.
Consider the following example:
$ pytest -v tests/test_demo.py
collected 6 items
tests/test_demo.py::test1 PASSED
tests/test_demo.py::test2 PASSED
tests/test_demo.py::test3 PASSED
tests/test_demo.py::test4 PASSED
tests/test_demo.py::test5 FAILED
tests/test_demo.py::test6 PASSED
$ pytest -v --lf tests/test_demo.py
collected 6 items / 5 deselected / 1 selected
tests/test_demo.py::test5 PASSED
You can use leak finder
On the first run will set the failed test as the "target" and will stop the session.
$ pytest -v --leak-finder tests/test_demo.py
collected 6 items
tests/test_demo.py::test1 PASSED
tests/test_demo.py::test2 PASSED
tests/test_demo.py::test3 PASSED
tests/test_demo.py::test4 PASSED
tests/test_demo.py::test5 FAILED
Leak finder: target set to tests/test_demo.py::test5
The second execution will run the first half of the tests passed before the target (let's say the half "A", composed by test1
and test2
).
If the target still fail, that path is followed deeper by dividing again. But in this example it passes, so we'll discard it.
$ pytest -v --leak-finder
collected 6 items / 3 deselected / 3 selected
tests/test_demo.py::test1 PASSED
tests/test_demo.py::test2 PASSED
tests/test_demo.py::test5 PASSED
Leak finder: We reach the target and nothing failed. Let's change the last half.
A new execution takes the group "B", i.e. test3
and test4
.
$ pytest -v --leak-finder
collected 6 items / 3 deselected / 3 selected
tests/test_demo.py::test3 PASSED
tests/test_demo.py::test4 PASSED
tests/test_demo.py::test5 FAILED
Leak finder: The group selected still fails. Let's do a new partition.
Lastly, a new partition is done and the group "B-A" is taken, i.e. test3
$ pytest -v --leak-finder
collected 6 items / 3 deselected / 3 selected
tests/test_demo.py::test3 PASSED
tests/test_demo.py::test5 FAILED
Leak finder: The group selected still fails. Let's do a new partition.
And there is it, test3
is the problematic test we were looking for!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pytest_leak_finder-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a5202fc0947696123a9b32960be916659a170aa588aea4d1747a49bf337cf272 |
|
MD5 | 7e7586150bd6e672f5024b826986b6c8 |
|
BLAKE2b-256 | 8f0f41dda462099b5e2fa011e6062d21718fbd69d27ecf0c70c7bae778231fd3 |