pytest-selective

Run affected Python tests. See why they matter.

Our README CI used 92.3% less runner time: 627s → 48s. Elapsed time fell from 211s to 26s, with docs, routing and package checks preserved. One successful same-revision pair, not a universal speedup. Inspect the runs and raw evidence.

pytest-selective is a Python test impact analysis tool for incremental testing with pytest. It follows dependencies through both sides of a Git change, selects the affected test files, and explains the result. Plan before collecting tests, importing your application, or starting databases and workers.

Built from the selector used in Gigaverse's backend, AI, and shared Python infrastructure. The API may change before 1.0. Provenance and extraction scope.

Quick tutorial · Adoption guide · GitHub Actions · Evidence and limits · Keep selection narrow

Run local edits before committing · Comparison with pytest-impacted

CI-esta now uses this selector in its own CI: a matched README-only pair used 49 instead of 105 runner-seconds (53.3% less) while keeping quality, routing and package checks. Its reports now compute dependency critical paths and slack. See the source runs and method.

Have your agent set it up

Paste this into your coding agent in the Python project you want to improve:

Read https://raw.githubusercontent.com/gigaverse-app/pytest-selective/main/skills/pytest-selective/SKILL.md
and use it to speed up this project's tests/CI. Preserve coverage, add routing
regressions, and prove the result with measured before/after evidence.

Install the skill or add AGENTS.md instructions. Works through existing Git/CLI tools; no new agent server is needed.

A small change should have an explainable test plan

shop/prices.py changed
    └── shop/checkout.py imports prices
            └── tests/test_checkout.py       RUN
                tests/test_health.py         unaffected
                tests/test_labels.py         unaffected

Shared conftest changed?                     RUN ALL THREE
Only the unreferenced README changed?        SKIP PYTEST

The executable tutorial creates real commits, introduces a pricing regression, and runs pytest on the selected files. One of three files is selected and catches the regression. This is a reproducible selection example, not a wall-time speedup claim.

Used by Due and pytest-obligation

Due and pytest-obligation plan every pull request with pytest-selective. Their conformance suites need PostgreSQL, Redis, Celery workers and real framework checkouts, so skipping an unaffected lane saves minutes. Measured on real GitHub Actions runs:

Each row is one successful pair on the same PR head and merge checkout: the selective run first, then a test:full control. Both include planning time. Single observations, not billed minutes or a promise for every PR. An affected lane still runs complete, with its conformance evidence checks. A Due docs-only PR also finished in 26s (91 runner-seconds), an observed run without a paired control (docs-routing evidence).

Adoption: Due #68 and pytest-obligation #93, each with its own lane map (.github/test-lanes.json). Run links, raw timestamps, method and limits, including Due's widget benchmark, which selection skips entirely.

Try it in two minutes

Requires Python 3.11+, Git 2.24+, and uv. Use a planner Python version at least as new as your source syntax; the CI action uses Python 3.14. Your test runtime can use a different Python version when planning runs separately. The optional audit recorder requires Python 3.11+ in the test environment.

git clone https://github.com/gigaverse-app/pytest-selective.git
cd pytest-selective
uv sync --locked
uv run python scripts/tutorial.py --output tutorial-report.json

transitive regression: scoped, 1/3 files; pytest exit 1 (expected)
documentation only: none, 0/3 files; pytest skipped
shared conftest: full, 3/3 files; pytest exit 1 (expected)

The nonzero pytest results are intentional: they prove the selected tests catch the introduced bug. The tutorial itself succeeds only when all expected outcomes hold. It uses a temporary repository and leaves your checkout untouched.

Install in your project

Until the first PyPI release, install from GitHub using Python 3.11+:

pip install "pytest-selective @ git+https://github.com/gigaverse-app/pytest-selective.git"

Pin a reviewed commit in CI. Then follow the configuration walkthrough to declare module roots, test roots, collection-policy fingerprints, and non-Python inputs.

pytest-selective affected --config .github/test-impact.json \
  --base origin/main --head HEAD --output selection.json \
  --explanation-output explanation.json

Planning reads committed Git trees. Uncommitted edits are outside the plan. The report distinguishes scoped, full, and none; an empty selection must skip pytest, because invoking pytest without paths runs the whole suite.

Why pytest-selective?

The CLI emits JSON for your runner. The composite action adds caching and falls back to complete test roots on operational planning failures. Invalid dependency declarations fail with a diagnostic that must be fixed.

What a green run means

Selection is only as complete as the dependency model. Static analysis cannot discover every dynamic import, plugin registration, environment dependency, or file read. Declare those edges, audit representative runs, and keep full-suite checkpoints. A green selected run is not proof that every test would pass.

Selection chooses which files are affected; your existing markers and CI policy still determine which tests may run. Suites that enforce complete collection or whole-suite coverage need full runs.

For expensive suites that must retain complete conformance evidence, use whole-lane selection. An affected test activates its entire lane, preserving the lane's existing tests and evidence checks.

Read the evidence, safety boundaries, and adoption checklist.

Choose the right tool

pytest-impacted offers a pytest-native --impacted workflow, including working-tree changes. pytest-selective focuses on committed revisions, inspectable JSON plans, runtime dependency audits, and planning before application startup. Both use static dependency analysis; choose based on your workflow, and validate selection against your own suite. See the lessons from pytest-impacted for portability and Git-resolution improvements adopted from that comparison.

Contribute

uv sync --locked
uv run pre-commit install
uv run pre-commit run --all-files
uv run pytest -q
uv build
uv run twine check --strict dist/*
uv run python scripts/smoke_distribution.py

Contributing · Documentation · Release process · Report a bug

_{CI-esta reports this repository's CI timing and renders pytest-selective's selection explanations next to it. It is optional; pytest-selective does not depend on it.}

How this project keeps CI fast

See the CI toolkit integration for lane selection, validated environment caches, required gates and real dependency critical-path reports.

Optimize the work that still has to run

Selection and execution solve different parts of slow CI. Once selection keeps the required tests, use CI-esta's critical-path report to find the job holding up the gate. Due's full conformance gate fell from 538 to 282 seconds with four isolated workers, preserving all 982 integration cases per Django variant and every validator. Obligation's gate fell from 506 to 422 seconds; its bottleneck then moved to Saleor.

These are one-pair execution measurements, separate from this project's affected-test selection results. Raw reports and matched test inventories show provenance, runner cost and the actual dependency paths. This repository also uses CI-esta's shared native Windows Git action while retaining full coverage on all six Python/platform lanes.