pytest-subset

A pytest plugin for deterministically sampling a subset of a heavily parametrized test's variants.

Installation

uv pip install pytest-subset

Usage

import pytest

@pytest.mark.subset(fraction=0.2)
@pytest.mark.parametrize("value", range(1000))
def test_something(value):
    ...

Running pytest normally will run ~20% of test_something's variants, chosen deterministically. Pass --full (or -F) to run every variant, e.g. in CI before a release.

Marker reference

@pytest.mark.subset(fraction=..., seed=None)

  • fraction: float in (0, 1]. Fraction of parametrized variants to keep. Optional if subset_default_fraction is set in your pyproject.toml.
  • seed: optional int. A seed value used to generate a pseudo-random slice of the parametric test space. Pins the selection for this test. If not provided, a default seed is generated based on the test module and Node ID. In either case, a given seed value will always generate the same subset of tests (i.e. deterministic)

CLI options

  • -F, --full — ignore every subset marker and run all variants.
  • --salt=SEED — controls the slice for tests that don't pin their own seed=. Accepts:
    • an integer, so different values give different, still-reproducible slices;
    • random, to pick a fresh seed each run. It's printed after the run so you can reuse it later.
import pytest

@pytest.mark.subset(fraction=0.2, seed=42)
@pytest.mark.parametrize("value", range(1000))
def test_something(value):
    ...

@pytest.mark.subset(fraction=0.2)
@pytest.mark.parametrize("value", range(1000))
def test_something_else(value):
    ...

test_something always uses seed 42 — an explicit seed= superceeds --salt every time. test_something_else has no seed of its own, so --salt decides its slice:

pytest --salt=22       # test_something_else samples using salt 22
pytest --salt=random   # a fresh seed is picked and printed after the run

Note that --salt=random picks each run's slice independently, so slices across runs aren't guaranteed to be disjoint.

Ini options

Set this under [tool.pytest.ini_options] in your pyproject.toml (or the equivalent section of pytest.ini/setup.cfg):

  • subset_default_fraction — fraction used by @pytest.mark.subset when it's applied bare, with no fraction= argument. This value is applied globally across the test suite. 
    [tool.pytest.ini_options]
subset_default_fraction = "0.3"

Then, in your test modules,

import pytest

@pytest.mark.subset  # 0.3 applied here by default.
@pytest.mark.parametrize("value", range(1000))
def test_something(value):
    ...

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for dotslashexecute from gravatar.com dotslashexecute

Credits

Author: cyber-paraphernalia

Requires

Python >=3.10

Provides Extra

dev

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest_subset-0.1.0.tar.gz (7.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pytest_subset-0.1.0-py3-none-any.whl (5.4 kB view details)

Uploaded Python 3

File details

Details for the file pytest_subset-0.1.0.tar.gz.

File metadata

  • Download URL: pytest_subset-0.1.0.tar.gz
  • Upload date:
  • Size: 7.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for pytest_subset-0.1.0.tar.gz
Algorithm Hash digest
SHA256 db473ac4b26001ff841dfe997537eee6adc8c0ca1ddfe04b4d4cbc644efdf61a
MD5 49e631d95a410cf4e7070f6c5873ce4b
BLAKE2b-256 242f54786ad5e8343e47f8a10cd9de98fb6826bb635afd3658ce91bc5cc40d25

See more details on using hashes here.

Provenance

The following attestation bundles were made for pytest_subset-0.1.0.tar.gz:

Publisher: publish.yml on cyber-paraphernalia/pytest-subset

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pytest_subset-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: pytest_subset-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 5.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for pytest_subset-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 28d085f270863af125171c305539f7602569f7e957367b099a6f575736ebc209
MD5 ceb99cf3f39ddf83003cbf04c8208d53
BLAKE2b-256 89ee4f08b71253d82c16a9744127eab787eeb195a9397abd047c2b31c9ce753f

See more details on using hashes here.

Provenance

The following attestation bundles were made for pytest_subset-0.1.0-py3-none-any.whl:

Publisher: publish.yml on cyber-paraphernalia/pytest-subset

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page