No project description provided
Project description
pytest-shard
Shards tests based on a hash of their test name enabling easy parallelism across machines, suitable for a wide variety of continuous integration services. Tests are split at the finest level of granularity, individual test cases, enabling parallelism even if all of your tests are in a single file (or even single parameterized test method).
Features
pytest-shard
aims for simplicity. When installed, simply run:
$ pytest --shard-id=I --num-shards=N
where I
is the index of this shard and N
the total number of shards. For example, to split tests across two machines:
# On machine 1:
$ pytest --shard-id=0 --num-shards=2
# On machine 2:
$ pytest --shard-id=1 --num-shards=2
The intended use case is for continuous integration services that allow you to run jobs in parallel. For CircleCI, enable parallelism and then use:
pytest --shard-id=${CIRCLE_NODE_INDEX} --num-shards=${CIRCLE_NODE_TOTAL}
On Travis, you must define the environment variables explicitly, but can use a similar approach.
Alternatives
pytest-xdist allows you to parallelize tests across cores on a single machine, and can also schedule tests on a remote machine. I use pytest-shard
to split tests across CI workers, and pytest-xdist
to parallelize across CPU cores within each worker.
pytest-shard
does not take into account the run time of tests, which can lead to suboptimal allocations. pytest-circleci-parallelized uses test run time, but can only split at the granularity of classes, and is specific to CircleCI.
Please open a PR if there are other promising alternatives that I have overlooked.
Installation
You can install pytest-shard
via pip
:
$ pip install pytest-shard
Contributions
Contributions are welcome. Test may be run using tox
.
License
This software is MIT licensed.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pytest_shard-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 407a1df385cebe1feb9b4d2e7eeee8b044f8a24f0919421233159a17c59be2b9 |
|
MD5 | 464dfb0e0c3c1fbffdaa193ca8a34610 |
|
BLAKE2b-256 | 657adbeb4c54e9fc3b59622f410091365f354a69cda1af10c3b83ac0ca6e6f4f |