pytest-hygiene 0.1.0

A cross-platform pytest plugin that audits test isolation and reports exactly what global state each test leaks.

pytest-hygiene

Find out which tests leak global state — and exactly what they leak. Cross-platform, no os.fork.

pytest-hygiene snapshots key global process state before and after every test's full lifecycle (setup + call + teardown), diffs the two, and tells you precisely what each test left dirty:

=============================== test hygiene ================================
3 state leak(s) detected across 2 test(s) (auditors: env, cwd, ..., random):

tests/test_api.py::test_uses_proxy
  LEAK [env] set env var 'HTTP_PROXY'='http://localhost:8080'
  LEAK [threads] non-daemon thread 'poller' still alive after test

tests/test_cli.py::test_runs_in_tmp
  LEAK [cwd] changed working directory: '/repo' -> '/tmp'

The problem: tests pass alone but fail together

You've hit it before. A test passes in isolation but fails when the suite runs — or worse, it causes an unrelated test three files over to fail. The culprit is almost always leaked global state: a test mutates something process-wide (an environment variable, the working directory, sys.path, a logging handler, the global RNG) and never restores it. The next test inherits the mess.

These failures are miserable to debug because the failing test isn't the guilty one, and the symptom (order-dependence) hides the cause (what state leaked).

Why existing tools fall short

Tool	Approach	Limitation
pytest-forked, pytest-isolate	Run each test in a forked subprocess	Fork-based — does not work on Windows. Heavy. Hides leaks rather than reporting them.
pytest-cleanslate	Bisects order-dependent failures at module granularity	Only tells you which module pair interacts; not what state leaked, and not per-test.

None of them answer the actual question: what did this test leave behind?

pytest-hygiene is different:

• Cross-platform. Pure snapshot/diff around the run-test protocol — no os.fork, no POSIX-only calls. Works on Windows, macOS and Linux (proven by the CI matrix).
• It names the leak. Not "these two modules interfere" but "test_foo left env var HTTP_PROXY set, added a logging handler to root, and spawned a non-daemon thread."
• Low overhead. Capturing is a handful of cheap dict/list copies per test.

Install

pip install pytest-hygiene

The plugin activates automatically (it registers a pytest11 entry point). There is nothing to import.

Quickstart

Just run your suite — auditing is on by default and stays quiet unless something leaks:

pytest

Leak a bit of state and you'll see the test hygiene section shown above. To turn leaks into hard failures (great for CI gating):

pytest --hygiene-strict

To temporarily disable auditing:

pytest --no-hygiene

What it audits

Each auditor snapshots one slice of global state. By default only leak-severity items are shown; warning/info items appear with --hygiene-verbose.

Auditor (aspect)	Detects	Default severity
env	Environment variables added, removed, or changed (os.environ). Ignores pytest's own PYTEST_CURRENT_TEST.	leak
cwd	Current working directory changed and not restored (os.getcwd()).	leak
syspath	Entries added to / removed from sys.path.	leak
sysmodules	A module in sys.modules replaced (same name, different object) or deleted = leak. Newly imported modules = info (shown only in verbose — first imports are usually benign).	leak / info
warnings	warnings.filters mutated and not restored.	leak
logging	Root logger level changed, handlers added/removed, or logging.disable() level changed.	leak
threads	Threads started during the test still alive at teardown. Non-daemon = leak; daemon = warning.	leak / warning
random	Global RNG state consumed/reseeded (random.getstate()), a determinism smell. If numpy is already imported, numpy.random global state too.	warning

numpy support is strictly optional: the random auditor only inspects numpy if it is already present in sys.modules. It never imports numpy itself, so there is zero cost if you don't use it.

Configuration

Every option is available both on the command line and as an ini setting (in pytest.ini, pyproject.toml, tox.ini, or setup.cfg). Command-line flags win over ini values.

Command-line

Flag	Effect
--hygiene / --no-hygiene	Enable / disable auditing (default: enabled).
--hygiene-strict	Report any test that leaks state as a failure (via its teardown phase, surfaced by pytest as an error). Only leak-severity items trigger this — not warnings/info.
--hygiene-aspects=env,cwd,...	Comma/space-separated auditor names to enable (default: all).
--hygiene-verbose	Also show info/warning items (new modules, RNG use, daemon threads), and print a clean-bill-of-health line when nothing leaks.

ini options

# pytest.ini
[pytest]
hygiene = true              ; bool, default true  — enable/disable
hygiene_strict = false      ; bool, default false — leaks become failures
hygiene_verbose = false     ; bool, default false — include info/warning items
hygiene_aspects =           ; args, default all   — e.g. "env cwd threads"
hygiene_allow =             ; linelist            — suppress known-benign leaks

Or in pyproject.toml:

[tool.pytest.ini_options]
hygiene = true
hygiene_strict = false
hygiene_aspects = ["env", "cwd", "threads"]
hygiene_allow = ["HTTP_PROXY", "env:CI", "sysmodules:my_app.*"]

The allowlist (hygiene_allow)

Some leaks are known and benign — a session-scoped fixture that intentionally sets an env var, a library that installs a logging handler on first import. Suppress them with hygiene_allow, one pattern per line:

• Bare pattern — matched against the leak's subject (env var name, module name, sys.path entry, thread name, cwd path). Supports shell-style globs:

  HTTP_PROXY          # exact env var
  HYG_*               # any env var / subject starting with HYG_
  

• Scoped pattern aspect:pattern — only applies within one auditor:

  env:CI              # allow the CI env var, but still flag it elsewhere
  sysmodules:my_app.*
  threads:worker-*
  

The scope prefix is only treated as a scope when it names a real auditor, so Windows paths like C:\Temp are matched literally (the leading C: is not mistaken for a scope).

Note: warnings leaks don't carry a single string subject, so they can't be allowlisted by pattern — disable the warnings aspect via --hygiene-aspects if you need to.

Known nuance: session/module/class-scoped fixtures

pytest-hygiene attributes a leak to the test during whose lifecycle the state changed. State set up by a session-, module-, or class-scoped fixture is deliberately not torn down after the first test that triggers it — so that first test can look like it "leaked" the fixture's state (an env var, an imported module, a logging handler).

This is expected, and v1 does not try to fully solve scope attribution. The escape hatch is hygiene_allow: allowlist the specific subject the shared fixture owns. For example, a session fixture that sets DATABASE_URL:

[tool.pytest.ini_options]
hygiene_allow = ["DATABASE_URL"]

How it works

1. A hook wrapper around pytest_runtest_protocol(item, nextitem) takes a before snapshot just before setup begins and stashes it on the item.
2. A hook wrapper around pytest_runtest_makereport takes the after snapshot once the teardown phase has fully finished, then diffs before vs. after.
3. Capturing around the entire setup + call + teardown lifecycle is deliberate: state that a fixture sets up and then correctly tears down nets to zero and is not reported. Only genuine leftovers count.
4. Leaks are collected per item.nodeid and printed in the test hygiene section at session end (pytest_terminal_summary). Under --hygiene-strict, a leaking test's teardown report is marked failed so the run's exit status reflects the leak.

Every snapshot is a shallow copy of a dict/list (or a tuple of identities), so per-test overhead is minimal.

Development

git clone https://github.com/az-pz/pytest-hygiene
cd pytest-hygiene
pip install -e .[dev]   # installs pytest + numpy (to exercise the optional path)
pytest -q

The test suite dogfoods the plugin: it runs with pytest-hygiene active and asserts our own tests leak nothing.

Releasing

Releases publish to PyPI automatically via the release.yml workflow using PyPI Trusted Publishing (OIDC) — no API tokens are stored.

One-time setup — on PyPI, go to Account → Publishing → Add a pending publisher and register:

Field	Value
Owner	az-pz
Repository	pytest-hygiene
Workflow name	release.yml
Environment	pypi

To cut a release:

1. Bump version in pyproject.toml (e.g. 0.1.0 → 0.2.0).
2. Commit, then tag and push:

   git tag v0.2.0
   git push origin v0.2.0
   

3. The workflow runs the test suite, builds the sdist + wheel, verifies the tag matches the package version, and publishes to https://pypi.org/project/pytest-hygiene/.

PyPI versions are immutable — each release needs a unique, higher version number.

Roadmap / deferred

Ideas intentionally left for a future version:

• MockAuditor — detect unittest.mock patches that were started but never stopped.
• Signal-handler auditor — flag changed signal handlers.
• pytest-xdist aggregation — collate the summary across worker processes.

License

MIT — see LICENSE.