assert-pytest-fixture-is-requested

Assert that every pytest fixture is asked for by something.

A fixture runs only when something asks for it. Delete the last test that asks, and the fixture stays: it is collected into every session that loads its file, the tests written for it keep passing, and the coverage gate on its package stays green. Nothing shows the requester has gone.

A check that matches the fixture's name against the source cannot answer this. A fixture's name is an ordinary identifier that anything else may bind, so a local variable credits it and so does an unrelated function of the same name. Worse, matching a name cannot tell which definition a request resolves to, and it counts a request written inside a factory that nothing ever calls.

So this tool asks pytest. It runs a real collection, reads the fixtures off it, and adds the one thing a collection cannot see: getfixturevalue.

What counts as asking

pytest resolves five of the six ways itself, and the collection reports them:

• a test function taking the fixture as a parameter
• another fixture taking it as a parameter
• a pytest.mark.usefixtures mark naming it as a string
• a request.getfixturevalue call naming it as a string
• autouse=True, which asks on every in-scope test's behalf

The sixth is a request.getfixturevalue call on a variable, where the name arrives from somewhere else:

def create_log_group_tests(log_group_fixture: str) -> type:
    class TestLogGroup:
        def test_retention(self, request):
            log_group = request.getfixturevalue(log_group_fixture)

Following that argument back to its call sites is interprocedural analysis. This tool takes the blunt instrument instead: where a getfixturevalue call is handed anything but a string literal, every string constant anywhere in the trees counts as a possible request. That over-credits — a dictionary key spelled like a fixture will spare a dead one — and the miss is the right way to be wrong, since a fixture wrongly kept costs a line and a fixture wrongly deleted costs a red suite. It degrades to nothing where a repository makes no such call.

Why a collection rather than a parse

Two things only pytest knows.

Which definition a request resolves to. Where conftests at two levels publish the same fixture name, pytest hands each item the one visible to it. A name match credits both, so a definition that every requester shadows looks alive.

Whether the asking is reachable. A parameter of a test method inside a factory nothing calls is not a request, because the class is never built and the method never becomes a test.

Install

pip install assert-pytest-fixture-is-requested

Use

assert-pytest-fixture-is-requested test lib

The trees are collected and read. Because the tool runs a real pytest collection, it must run where the target's tests import: install the test dependencies and set PYTHONPATH as the suite expects.

Options

• --defined-in TREE — a tree the own-tests exclusion applies to. Repeatable.
• --dont-search-in PATTERN — a path whose requests do not credit a fixture of the matching package, with {package} standing for the package name. Repeatable.
• --import-mode MODE — the pytest import mode. Defaults to importlib, which reads a whole tree whose test files repeat a basename.
• --rootdir PATH — the directory reported paths are expressed against. Defaults to the working directory.
• --pytest-argument ARGUMENT — a further argument for the collection, such as --pytest-argument=--confcutdir=test. Repeatable.
• --annotate — print each finding as a GitHub Actions error annotation.
• --quiet, --count, --verbose — choose the output.
• --fail-fast, --warn-only — stop at the first finding, or always exit 0.

The own-tests exclusion

A suite written to exercise a fixture asks for it, so crediting that suite makes every such fixture look used. Name the trees the exclusion applies to and the shape of the suite that does not count:

assert-pytest-fixture-is-requested test lib \
  --defined-in lib/python \
  --dont-search-in 'test/lib/python/test_{package}'

A fixture published from lib/python/<package>/ is then not credited by a request from test/lib/python/test_<package>/. A fixture defined under test/ is credited by a request from anywhere, its own file included, because the conftest that publishes it and the suite that asks for it are the same subtree by design.

Exit codes

• 0 — every fixture is asked for
• 1 — at least one fixture is asked for by nothing
• 2 — a tree was missing, a file was unreadable, or the collection failed

A collection error exits 2 rather than 1, because a file that fails to import registers none of its requests and would make live fixtures look dead.

GitHub Actions

- uses: 10U-Labs/assert-pytest-fixture-is-requested@latest
  with:
    trees: test lib
    defined-in: lib/python
    dont-search-in: test/lib/python/test_{package}
    verbose: "true"

Limitations

• The tool must run where the target's tests import cleanly.
• A dynamic getfixturevalue call makes every string constant a possible request, which deliberately over-credits.
• Fixtures registered by pytest itself and by installed plugins are ignored, because they are written outside the trees handed over.

Licence

Apache-2.0.