skippy-cov
Repository: mgaitan/skippy-cov. Supports Python 3.12–3.15. Development uses uv, Ruff, ty and optional prek hooks; see Contributing.
Overview
skippy-cov is a tool that optimizes your Python testing workflow by filtering tests based on code changes. It takes a diff file from git and a .coverage database (generated by coverage.py) to identify and run only the tests relevant to the modified code. This significantly reduces testing time and resources.
Usage
To use this, you first need to generate the following required files:
- Run Coverage Analysis: Generate a
.coveragedatabase usingcoverage.pyviapytest-covwith test contexts
pytest --cov=myproj --cov-context=test myproj/tests/
Then change your code and run
pytest --skippy-cov
In this case, skippy-cov will analyze the changes and run only the tests that are relevant to the modified code.
You can also use skippy-cov as a CLI on its own.
skippy-cov
that will list the tests selected.
Configuration
The inputs for skippy-cov are the coverage file (default: .coverage) and the diff information. By default, the diff is generated in the same way as a pull request (PR) on GitHub: only the changes introduced by your branch compared to the target branch are considered. This is done using the git diff <target>...HEAD (triple-dot) syntax.
You can specify the coverage file and a branch or ref to diff against (e.g., the main branch):
skippy-cov --diff main --coverage-file .coverage
or equivalently
pytest --skippy-cov --skippy-cov-diff main --skippy-cov-coverage-file .coverage
You can also provide a diff file:
git diff > changes.diff
skippy-cov --diff changes.diff --coverage-file .coverage
Single refs always use <ref>...HEAD; explicit .. and ... ranges are preserved.
Git refs compare committed changes. Use a patch file as above to include working-tree
changes. Default-branch detection uses local refs and does not contact the remote.
Without origin/HEAD, it checks local main/master branches, then fetched
origin/main/origin/master refs. Fetch the required refs before selecting tests
in a shallow CI checkout; the comparison still needs a common ancestor.
CLI output is deterministic. --format lines emits one target per line;
--format json maps each test module to its selected node IDs, where an empty list
means the entire module. Entries for the same module are merged. The default
format quotes paths for copying into a shell command; prefer JSON for automation.
--relative-to accepts files and directories without mutating the selection;
--strip-prefix requires a single scope. Pytest's legacy
--skippy-cov-strip-prefix option is deprecated because the plugin preserves the
caller's resolved scopes and exact node IDs.
If you omit the --diff argument, it will default to the main branch as determined by your git remote (usually "main" or "master").
Advanced:
You may pass any valid git diff refspec to --diff, including triple-dot syntax (e.g., main...HEAD or origin/master...feature-branch). If you provide a triple-dot ref, it will be used as-is, giving you full control over the comparison range.
See skippy-cov --help for more information.
Coverage baseline
Run skippy-cov from the repository root. Invocation from a Git subdirectory is rejected with an explicit diagnostic. The pytest plugin warns and keeps the original test selection when baseline validation fails.
The baseline must be a readable coverage database with pytest test contexts,
generated using --cov-context=test. Missing, empty, corrupt or context-free
data is an error; the CLI exits with status 2 rather than printing an apparently
valid empty selection.
Validation does not establish baseline completeness or revision provenance. The caller must use data from a successful complete test run, retain its commit SHA, and compare against that revision, including intervening changes to the base branch. A readable database alone does not guarantee a trustworthy selection.
Prefer portable coverage paths when collecting data in another checkout or Docker:
[tool.coverage.run]
relative_files = true
core = "ctrace"
If a baseline already contains absolute paths from another checkout, provide its original root explicitly:
skippy-cov --coverage-file baseline/.coverage --source-root /fierro
pytest --skippy-cov --skippy-cov-coverage-file baseline/.coverage --skippy-cov-source-root /fierro
Paths are matched relative to the checkout root, never guessed from basenames.
Every recorded source path must map to that root. Mixed baselines from multiple
absolute checkout roots are rejected rather than silently losing test consumers;
combine them with coverage's path aliases or collect relative paths first.
On Python 3.14+, coverage defaults to
sysmon, which does not support dynamic test contexts; collect the baseline with
core = "ctrace" as shown above.
Conservative selection and fallback
By default, selection runs whole test modules. Pytest discovers unittest classes, inherited methods, custom function names and parameterizations; skippy-cov does not rebuild their node IDs with a separate AST visitor.
Individual test selection (opt-in)
The default granularity is module. Individual selection is an opt-in heuristic:
caches and state shared between tests can hide consumers from coverage. Keep
module selection as the default and run the complete suite before merging.
To select individual tests, use:
pytest --skippy-cov --skippy-cov-diff=BASELINE_SHA..HEAD --skippy-cov-granularity=test
skippy-cov --diff BASELINE_SHA..HEAD --granularity test --format json
The coverage database must have been collected at BASELINE_SHA, with
--cov-context=test. Its revision cannot be inferred from a coverage database
alone; providing coverage from a different commit can produce an incorrect
selection. A single Git ref uses its merge base with HEAD, while an explicit
BASELINE_SHA..HEAD range uses that exact baseline. The new revision must be
HEAD. With a patch file, provide --base-ref BASELINE_SHA to the CLI or
--skippy-cov-base-ref=BASELINE_SHA to pytest so the old source is available.
This mode parses function, async-function, method, nested-function, and class
ranges from the old source without executing it. A function-body change selects
the tests recorded anywhere in that function's baseline runtime body, including
consumers in other test files. It uses old line numbers for coverage, so shifted
definitions and insertions into previously uncovered branches remain eligible.
Unrelated sibling functions and methods do not share a selection.
Decorated functions and methods inside decorated scopes require the original
selection: a decorator such as functools.lru_cache can return a value without
executing the recorded function body for every consumer.
Every new test is included independently of coverage and source impact.
Changed test modules run in full, including new functions, inherited methods,
and new parameter cases. Removing lines without known runtime coverage, including
parameter declarations, requires the original selection. The plugin keeps pytest's
original collection and then deselects unaffected tests, respecting requested paths, exact node IDs, -k,
and -m. For an affected parameterized test it runs all currently collected
cases within that original scope. The CLI resolves current IDs through an
isolated pytest collection; pytest must be installed for this mode. It keeps
stdout suitable for the selected output format and protects the baseline
coverage database during collection. Test paths/discovery configuration apply;
the CLI's collection clears execution addopts and PYTEST_ADDOPTS. Apply
execution filters to the eventual pytest invocation, or use the plugin to keep
the full original pytest argument semantics.
New standalone functions, signature/decorator changes, class attributes/bases, module declarations, ambiguous scope matches, unknown/import-time coverage, and shared setup/teardown require the original selection. Recorded file consumers cannot establish every import-time dependency, such as a constant imported by another test module. The plugin preserves the original selection with a diagnostic; the CLI reports that it cannot select safely. New source files and non-Python changes retain that full-selection fallback. The test suite compares individual, module, and full runs with real coverage, including a shared-session-fixture regression where narrowing could hide a failing second consumer. Historical coverage remains an impact heuristic.
Module selection rules
Changes to existing source function bodies with recorded execution under a
test's run phase select the consuming test modules. If a source diff also
changes imports, declarations, or otherwise has unknown impact,
skippy-cov preserves the original selection. Shared fixtures may
execute only for the first test that requests them, so those contexts cannot
identify every test that reuses their results. A file-level fallback involving
setup/teardown therefore requires the original pytest selection, as does a file
with no recorded consumers. This does not prevent selection for validated
function-body changes covered only during run in the same source file.
Changes to test modules select the whole module when edits stay within test
functions, including newly added tests and pytest.mark.parametrize cases.
Removing lines with unknown or import-time coverage requires the original
selection; comments and blank-looking text inside string literals are code.
Imports, other decorators, fixtures, and recorded setup/teardown execution
still require the original selection because they can affect shared test setup.
Conftest files, non-Python inputs, new/deleted/renamed files, and unknown
coverage also require the original selection. Recorded consumers of inherited
tests are included alongside the edited module. Separate edit blocks are
validated independently, even when Git groups them into one hunk.
An empty diff or no affected tests inside the caller's scope exits with pytest
status 5, not a successful test run.
The pytest plugin warns and keeps the original arguments when the baseline or
impact is unknown. The CLI exits with status 2 for those cases: callers must run
their full test command instead of treating an error as an empty selection.
Explicit node IDs and absolute paths passed to pytest retain their scope.
An explicit --rootdir does not change the base used for selected module paths.
Unresolvable Git refs also warn and preserve the original pytest selection.
Historical coverage is a heuristic, not a proof of all dependencies. Reuse data only from a successful complete run, keep its commit SHA alongside the artifact, and compare against that commit, including intervening changes to the base branch. Dynamic imports, cross-test state and shared fixtures still warrant periodic full runs and a complete suite before merging.
Contributing
See CONTRIBUTING.md for information on how to contribute to the project.
License
See LICENSE for licensing information.
Metadata
Release files for skippy-cov 0.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| skippy_cov-0.4.0.tar.gz | 54.5 kB | With provenance | Details for skippy_cov-0.4.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| skippy_cov-0.4.0-py3-none-any.whl | Python 3 | none | any | Details for skippy_cov-0.4.0-py3-none-any.whl |
Total release size: 75.2 kB
Release files / skippy_cov-0.4.0.tar.gz
| Download URL | skippy_cov-0.4.0.tar.gz |
|---|---|
| Size | 54.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e3a78d57dc32656b424bc8678f8d6dc567b64b9029b0b35dfcad01549095ac26
|
|
BLAKE2b-256 checksum How to use checksums |
3e502eb53a8d897ae8f30d326da4c18636ba7a833b70457121227bc775eab49d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.
Transparency logRelease files / skippy_cov-0.4.0-py3-none-any.whl
| Download URL | skippy_cov-0.4.0-py3-none-any.whl |
|---|---|
| Size | 20.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
953e9c0d80e33a0bc4f4f413f4b08f4558f359d3b920cb61486c65edabeeb921
|
|
BLAKE2b-256 checksum How to use checksums |
dd4a06c183cab88a029dbaedd2ffbcc545db1afdc1fa8bc41f60d12102388919
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.
Transparency logSecurity for skippy-cov 0.4.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for skippy_cov-0.4.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.