Skip to main content

skippy-cov

Repository: mgaitan/skippy-cov. Supports Python 3.12–3.15. Development uses uv, Ruff, ty and optional prek hooks; see Contributing.

Overview

skippy-cov is a tool that optimizes your Python testing workflow by filtering tests based on code changes. It takes a diff file from git and a .coverage database (generated by coverage.py) to identify and run only the tests relevant to the modified code. This significantly reduces testing time and resources.

Usage

To use this, you first need to generate the following required files:

  • Run Coverage Analysis: Generate a .coverage database using coverage.py via pytest-cov with test contexts
    pytest --cov=myproj --cov-context=test myproj/tests/

Then change your code and run

pytest --skippy-cov

In this case, skippy-cov will analyze the changes and run only the tests that are relevant to the modified code.

You can also use skippy-cov as a CLI on its own.

skippy-cov

that will list the tests selected.

Configuration

The inputs for skippy-cov are the coverage file (default: .coverage) and the diff information. By default, the diff is generated in the same way as a pull request (PR) on GitHub: only the changes introduced by your branch compared to the target branch are considered. This is done using the git diff <target>...HEAD (triple-dot) syntax.

You can specify the coverage file and a branch or ref to diff against (e.g., the main branch):

skippy-cov --diff main --coverage-file .coverage

or equivalently

pytest --skippy-cov --skippy-cov-diff main --skippy-cov-coverage-file .coverage

You can also provide a diff file:

git diff > changes.diff
skippy-cov --diff changes.diff --coverage-file .coverage

Single refs always use <ref>...HEAD; explicit .. and ... ranges are preserved. Git refs compare committed changes. Use a patch file as above to include working-tree changes. Default-branch detection uses local refs and does not contact the remote. Without origin/HEAD, it checks local main/master branches, then fetched origin/main/origin/master refs. Fetch the required refs before selecting tests in a shallow CI checkout; the comparison still needs a common ancestor.

CLI output is deterministic. --format lines emits one target per line; --format json maps each test module to its selected node IDs, where an empty list means the entire module. Entries for the same module are merged. The default format quotes paths for copying into a shell command; prefer JSON for automation. --relative-to accepts files and directories without mutating the selection; --strip-prefix requires a single scope. Pytest's legacy --skippy-cov-strip-prefix option is deprecated because the plugin preserves the caller's resolved scopes and exact node IDs.

If you omit the --diff argument, it will default to the main branch as determined by your git remote (usually "main" or "master").

Advanced: You may pass any valid git diff refspec to --diff, including triple-dot syntax (e.g., main...HEAD or origin/master...feature-branch). If you provide a triple-dot ref, it will be used as-is, giving you full control over the comparison range.

See skippy-cov --help for more information.

Coverage baseline

Run skippy-cov from the repository root. Invocation from a Git subdirectory is rejected with an explicit diagnostic. The pytest plugin warns and keeps the original test selection when baseline validation fails.

The baseline must be a readable coverage database with pytest test contexts, generated using --cov-context=test. Missing, empty, corrupt or context-free data is an error; the CLI exits with status 2 rather than printing an apparently valid empty selection.

Validation does not establish baseline completeness or revision provenance. The caller must use data from a successful complete test run, retain its commit SHA, and compare against that revision, including intervening changes to the base branch. A readable database alone does not guarantee a trustworthy selection.

Prefer portable coverage paths when collecting data in another checkout or Docker:

[tool.coverage.run]
relative_files = true
core = "ctrace"

If a baseline already contains absolute paths from another checkout, provide its original root explicitly:

skippy-cov --coverage-file baseline/.coverage --source-root /fierro
pytest --skippy-cov --skippy-cov-coverage-file baseline/.coverage --skippy-cov-source-root /fierro

Paths are matched relative to the checkout root, never guessed from basenames. Every recorded source path must map to that root. Mixed baselines from multiple absolute checkout roots are rejected rather than silently losing test consumers; combine them with coverage's path aliases or collect relative paths first. On Python 3.14+, coverage defaults to sysmon, which does not support dynamic test contexts; collect the baseline with core = "ctrace" as shown above.

Conservative selection and fallback

By default, selection runs whole test modules. Pytest discovers unittest classes, inherited methods, custom function names and parameterizations; skippy-cov does not rebuild their node IDs with a separate AST visitor.

Individual test selection (opt-in)

The default granularity is module. Individual selection is an opt-in heuristic: caches and state shared between tests can hide consumers from coverage. Keep module selection as the default and run the complete suite before merging. To select individual tests, use:

pytest --skippy-cov --skippy-cov-diff=BASELINE_SHA..HEAD --skippy-cov-granularity=test
skippy-cov --diff BASELINE_SHA..HEAD --granularity test --format json

The coverage database must have been collected at BASELINE_SHA, with --cov-context=test. Its revision cannot be inferred from a coverage database alone; providing coverage from a different commit can produce an incorrect selection. A single Git ref uses its merge base with HEAD, while an explicit BASELINE_SHA..HEAD range uses that exact baseline. The new revision must be HEAD. With a patch file, provide --base-ref BASELINE_SHA to the CLI or --skippy-cov-base-ref=BASELINE_SHA to pytest so the old source is available.

This mode parses function, async-function, method, nested-function, and class ranges from the old source without executing it. A function-body change selects the tests recorded anywhere in that function's baseline runtime body, including consumers in other test files. It uses old line numbers for coverage, so shifted definitions and insertions into previously uncovered branches remain eligible. Unrelated sibling functions and methods do not share a selection. Decorated functions and methods inside decorated scopes require the original selection: a decorator such as functools.lru_cache can return a value without executing the recorded function body for every consumer.

Every new test is included independently of coverage and source impact. Changed test modules run in full, including new functions, inherited methods, and new parameter cases. Removing lines without known runtime coverage, including parameter declarations, requires the original selection. The plugin keeps pytest's original collection and then deselects unaffected tests, respecting requested paths, exact node IDs, -k, and -m. For an affected parameterized test it runs all currently collected cases within that original scope. The CLI resolves current IDs through an isolated pytest collection; pytest must be installed for this mode. It keeps stdout suitable for the selected output format and protects the baseline coverage database during collection. Test paths/discovery configuration apply; the CLI's collection clears execution addopts and PYTEST_ADDOPTS. Apply execution filters to the eventual pytest invocation, or use the plugin to keep the full original pytest argument semantics.

New standalone functions, signature/decorator changes, class attributes/bases, module declarations, ambiguous scope matches, unknown/import-time coverage, and shared setup/teardown require the original selection. Recorded file consumers cannot establish every import-time dependency, such as a constant imported by another test module. The plugin preserves the original selection with a diagnostic; the CLI reports that it cannot select safely. New source files and non-Python changes retain that full-selection fallback. The test suite compares individual, module, and full runs with real coverage, including a shared-session-fixture regression where narrowing could hide a failing second consumer. Historical coverage remains an impact heuristic.

Module selection rules

Changes to existing source function bodies with recorded execution under a test's run phase select the consuming test modules. If a source diff also changes imports, declarations, or otherwise has unknown impact, skippy-cov preserves the original selection. Shared fixtures may execute only for the first test that requests them, so those contexts cannot identify every test that reuses their results. A file-level fallback involving setup/teardown therefore requires the original pytest selection, as does a file with no recorded consumers. This does not prevent selection for validated function-body changes covered only during run in the same source file.

Changes to test modules select the whole module when edits stay within test functions, including newly added tests and pytest.mark.parametrize cases. Removing lines with unknown or import-time coverage requires the original selection; comments and blank-looking text inside string literals are code. Imports, other decorators, fixtures, and recorded setup/teardown execution still require the original selection because they can affect shared test setup. Conftest files, non-Python inputs, new/deleted/renamed files, and unknown coverage also require the original selection. Recorded consumers of inherited tests are included alongside the edited module. Separate edit blocks are validated independently, even when Git groups them into one hunk. An empty diff or no affected tests inside the caller's scope exits with pytest status 5, not a successful test run.

The pytest plugin warns and keeps the original arguments when the baseline or impact is unknown. The CLI exits with status 2 for those cases: callers must run their full test command instead of treating an error as an empty selection. Explicit node IDs and absolute paths passed to pytest retain their scope. An explicit --rootdir does not change the base used for selected module paths. Unresolvable Git refs also warn and preserve the original pytest selection.

Historical coverage is a heuristic, not a proof of all dependencies. Reuse data only from a successful complete run, keep its commit SHA alongside the artifact, and compare against that commit, including intervening changes to the base branch. Dynamic imports, cross-test state and shared fixtures still warrant periodic full runs and a complete suite before merging.

Contributing

See CONTRIBUTING.md for information on how to contribute to the project.

License

See LICENSE for licensing information.

Metadata

Release files for skippy-cov 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for skippy-cov 0.4.0
File Size Uploaded Provenance Actions
skippy_cov-0.4.0.tar.gz 54.5 kB With provenance Details for skippy_cov-0.4.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for skippy-cov 0.4.0
File Interpreter ABI Platform Actions
skippy_cov-0.4.0-py3-none-any.whl Python 3 none any Details for skippy_cov-0.4.0-py3-none-any.whl

Total release size: 75.2 kB

Release files / skippy_cov-0.4.0.tar.gz

Download URL skippy_cov-0.4.0.tar.gz
Size 54.5 kB
Tags Source
SHA-256 checksum
How to use checksums
e3a78d57dc32656b424bc8678f8d6dc567b64b9029b0b35dfcad01549095ac26
BLAKE2b-256 checksum
How to use checksums
3e502eb53a8d897ae8f30d326da4c18636ba7a833b70457121227bc775eab49d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

Release files / skippy_cov-0.4.0-py3-none-any.whl

Download URL skippy_cov-0.4.0-py3-none-any.whl
Size 20.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
953e9c0d80e33a0bc4f4f413f4b08f4558f359d3b920cb61486c65edabeeb921
BLAKE2b-256 checksum
How to use checksums
dd4a06c183cab88a029dbaedd2ffbcc545db1afdc1fa8bc41f60d12102388919
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

Security for skippy-cov 0.4.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for skippy_cov-0.4.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page