pytest plugin help to find coupled tests
Project description
pytest-sherlock
Pytest plugin which help to find coupled tests.
Sometimes we have coupled tests which depend from ordering
For example:
- PASSES
tests/exmaple/test_all_read.py tests/exmaple/test_b_modify.py tests/exmaple/test_c_delete.py
- FAILED
tests/exmaple/test_c_delete.py tests/exmaple/test_b_modify.py tests/exmaple/test_all_read.py
In this case pretty simple to detect coupled tests, but if we have >=1k tests which called before it will hard
Content:
Install
pip install pytest-sherlock
how to use:
pytest tests/exmaple/test_c_delete.py tests/exmaple/test_b_modify.py tests/exmaple/test_all_read.py --flaky-test="test_read_params" -vv -x
Plugin didn't run all tests, it try to find some possible guilty test and will run first
======================================================================================== test session starts ========================================================================================
collected 3 items
Try to find coupled tests:
tests/exmaple/test_b_modify.py::test_modify_random_param PASSED [ 33%]
tests/exmaple/test_all_read.py::test_read_params FAILED [ 66%]
tests/exmaple/test_c_delete.py::test_delete_random_param PASSED [100%]
============================================================================================= FAILURES ==============================================================================================
Also you can use pytest -x
or --exitfirst
======================================================================================== test session starts ========================================================================================
collected 3 items
Try to find coupled tests:
tests/exmaple/test_b_modify.py::test_modify_random_param PASSED [ 33%]
tests/exmaple/test_all_read.py::test_read_params FAILED [ 66%]
============================================================================================= FAILURES ==============================================================================================
TODO
I have a couple ideas, how to improve finder coupled tests:
- use AST for detect common peace of code (variables, functions, etc...)
- run not all tests (binary search algorithm)
- Also need to add tests for it =)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pytest-sherlock-0.0.1.tar.gz
(3.7 kB
view hashes)
Built Distribution
Close
Hashes for pytest_sherlock-0.0.1-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f040c609317a6d2d3e58db31353f6b97c4795af42584c5dfe9674acd84b7b346 |
|
MD5 | 63e9566814b96dc746c2f6043d193b58 |
|
BLAKE2b-256 | 2f8ca64a92d2647ca86004e13677cb295f822fd424636a87767718723c8c5a5a |