Skip to main content

Pytest resource guard infrastructure for enforcing test marks on external resource usage

Project description

resource-guards

Pytest infrastructure for enforcing that tests declare their external resource usage via marks.

Resource guards catch two classes of bugs:

  • Missing marks: a test calls an external resource without the corresponding @pytest.mark.<resource>. The guard fails the test with a clear message.
  • Superfluous marks: a test carries a resource mark but never actually invokes the resource. The guard fails the test so the mark doesn't rot.

How it works

There are two guard mechanisms, covering CLI binaries and Python SDKs respectively.

Binary guards create wrapper scripts that shadow the real binary on PATH. During a test, the wrapper checks environment variables to decide whether the test is allowed to use the binary. If not, it records a tracking file and exits 127. If yes, it records a tracking file and delegates to the real binary.

SDK guards monkeypatch a chokepoint in a Python SDK. The monkeypatched function calls enforce_sdk_guard(), which checks the same environment variables and either raises ResourceGuardViolation or records a tracking file.

Both mechanisms use per-test tracking files so the makereport hook can detect violations even when the test swallows errors or handles non-zero exit codes.

Built-in guards

mngr provides Docker guards out of the box (in imbue.mngr.register_guards_docker), and the mngr_modal plugin provides Modal guards (in imbue.mngr_modal.register_guards). These are registered automatically in each project's conftest.py.

Setup

In your conftest.py, register each resource you want to guard. You need two things per resource: a marker (so pytest knows about the mark) and a guard (so the enforcement hooks are installed).

# conftest.py
from imbue.resource_guards.resource_guards import (
    register_resource_guard,
    start_resource_guards,
    stop_resource_guards,
)

# Register a binary guard for tmux
register_resource_guard("tmux")

def pytest_configure(config):
    config.addinivalue_line("markers", "tmux: marks tests that use tmux")

def pytest_sessionstart(session):
    start_resource_guards(session)

def pytest_sessionfinish(session, exitstatus):
    stop_resource_guards()

Then mark your tests:

import pytest

@pytest.mark.tmux
def test_agent_creates_tmux_session():
    ...

Writing a custom SDK guard

You can guard any Python SDK by registering an install/cleanup pair:

from imbue.resource_guards.resource_guards import enforce_sdk_guard
from imbue.resource_guards.resource_guards import register_sdk_guard

_originals = {}

def _install():
    _originals["send"] = SomeClient.send
    SomeClient.send = _guarded_send

def _cleanup():
    if "send" in _originals:
        SomeClient.send = _originals["send"]
        _originals.clear()

def _guarded_send(self, *args, **kwargs):
    enforce_sdk_guard("my_sdk")
    return _originals["send"](self, *args, **kwargs)

register_sdk_guard("my_sdk", _install, _cleanup)

The key requirement is that your monkeypatch calls enforce_sdk_guard("my_sdk") at the SDK's chokepoint -- the single method through which all external calls flow.

Compatibility with pytest-xdist

Binary guards work transparently with xdist. The controller process creates the wrapper scripts and modifies PATH; workers inherit both via environment variables. SDK guards are installed independently in each process (controller and workers), since monkeypatches are process-local.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

resource_guards-0.1.3.tar.gz (16.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

resource_guards-0.1.3-py3-none-any.whl (9.2 kB view details)

Uploaded Python 3

File details

Details for the file resource_guards-0.1.3.tar.gz.

File metadata

  • Download URL: resource_guards-0.1.3.tar.gz
  • Upload date:
  • Size: 16.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for resource_guards-0.1.3.tar.gz
Algorithm Hash digest
SHA256 dc00cb76c1428de05280c0462ca0f584bc1966c422057cd433c53b4f414e9880
MD5 72f2a8d7dc4dbb26c85178fce2ce7734
BLAKE2b-256 b6b131cc6ae5f4afced758e46ec185573175533209d7fdef48d6c4e432348393

See more details on using hashes here.

Provenance

The following attestation bundles were made for resource_guards-0.1.3.tar.gz:

Publisher: publish.yml on imbue-ai/mngr

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file resource_guards-0.1.3-py3-none-any.whl.

File metadata

File hashes

Hashes for resource_guards-0.1.3-py3-none-any.whl
Algorithm Hash digest
SHA256 dd06d359777e85a3f3c5296bc2203ab05e618d7bd9c3840f024f9b48106803fc
MD5 94246f6d0b062f3eafc1a1ef4fd11447
BLAKE2b-256 a8fffd6a9217665a9b7933ce92e5b2997a7df8eca3efecc154a458cf382d590c

See more details on using hashes here.

Provenance

The following attestation bundles were made for resource_guards-0.1.3-py3-none-any.whl:

Publisher: publish.yml on imbue-ai/mngr

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page