This release is a pre-release and may not be stable for production use.
Introduction
Pytest plugin to repeat tests and pass if the test passes a number of times out of a total.
Purpose
I want to leverage pytest for model evaluation in a way that can be readily incorporated into CI/CD flows.
(Also consider giving pytest-repeat a look - I wrote pytest-repeated for statistical testing, as in, there are situations where one or two failures out of a hundred is acceptable.)
Installation
pip install pytest-repeated
🚀 Usage Example
@pytest.mark.repeated(times=4, threshold=2)
def test_example_fail_once():
import random
assert random.choice([True, False]) # may pass or fail
This test will run four times and pass if we get True in at least two of the four iterations.
🛠️ Development
The only requirement is 🐳 Docker.
(The .devcontainer and tasks.json are prepared assuming a *nix system, but if you know the commands, this will work on Windows, too.)
- Clone the repo.
- Branch out.
- Open in "devcontainer" on VS Code and start developing. Run
pytestunderteststo test. - Akternatively, if you are a fan of Test-Driven Development like me, you can run the tests without getting on a container.
.vscode/tasks.jsonhas the command to do so, but it's also listed here:
docker compose -f tests/docker-compose.yaml up --build --abort-on-container-exit --exit-code-from test
- When satisfied, push and open a PR. The pipeline will publish automatically when your PR is merged.
Future Plans
- Better statistical measures
- Optimized testing - stop conditions.
- Sequential testing.
- Ability to set the seed.
- Report and fail on speed
1 maintainer
Metadata
1 maintainer
Release files for pytest-repeated 0.2.3.dev202511270022
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| pytest_repeated-0.2.3.dev202511270022.tar.gz | 7.2 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pytest_repeated-0.2.3.dev202511270022-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 13.5 kB
Release files / pytest_repeated-0.2.3.dev202511270022.tar.gz
| Download URL | pytest_repeated-0.2.3.dev202511270022.tar.gz |
|---|---|
| Size | 7.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
20b26e9d6acd293207b36d2424a29c0eb8c63ff902d1cc92061309c0eec5c9dd
|
|
BLAKE2b-256 checksum How to use checksums |
e4c59dcfda6dd4b111470653ba9a1e556fe55d748d643b2901baa34928dd796b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 27, 2025.
Transparency logRelease files / pytest_repeated-0.2.3.dev202511270022-py3-none-any.whl
| Download URL | pytest_repeated-0.2.3.dev202511270022-py3-none-any.whl |
|---|---|
| Size | 6.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
17e1db3351e0e76057f41d8c7af0afb25ab793a7a197dd4c9a8eabde8fd2e633
|
|
BLAKE2b-256 checksum How to use checksums |
07af22ccfe22098d9a3c8565b21b1f06ff65bf15a4a0952a1b05460a75108527
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 27, 2025.
Transparency log
