Skip to main content

pytest-vigil

Pytest Vigil

License: MIT Python GitHub last commit GitHub Release Date

Pytest Vigil is a reliability pytest plugin that enforces resource limits on your tests and kills them when they exceed those limits.

Why you might need this

  • Tests sometimes hang indefinitely due to deadlocks or infinite loops
  • Memory leaks crash your test runner or CI environment
  • CPU-intensive tests slow down your entire suite
  • You want to enforce maximum runtime for your CI pipeline
  • You need to identify which tests are resource hogs

✨ Features

  • Deadlock detection: Kill tests that hang with low CPU activity
  • Suite timeout: Stop the entire test run after a specified duration
  • CI scaling: Automatically relaxes limits by 2x in CI environments (configurable)
  • Retry mechanism: Re-run tests that fail due to resource violations
  • Detailed reports: JSON output showing CPU breakdown by process type (browser, renderer, etc.)
  • Per-test timeout: Layered timeout enforcement — kernel-level alarm backstop, C-level faulthandler diagnostics, and optional force-exit escalation for tests stuck in GIL-holding C extensions
  • Per-test memory limit: Kills tests that exceed memory limits
  • Per-test CPU limit: Kills tests that exceed CPU usage limits

🚀 Installation

uv add -D pytest-vigil
# or
pip install pytest-vigil

⚡ Quick Start

1. Protect against heavy tests Limit tests to 5 seconds, 512MB RAM, and 80% CPU:

pytest --vigil-timeout 5 --vigil-memory 512 --vigil-cpu 80

2. Prevent infinite CI hangs Kill the entire suite if it runs longer than 15 minutes:

pytest --vigil-session-timeout 900

3. Generate Reliability Report

pytest --vigil-json-report
# saved to .pytest_vigil/vigil_report.json

🛠 Usage & Configuration

CLI Options Reference

Option Default Description
--vigil-timeout None Max duration per test (seconds)
--vigil-memory None Max memory usage (MB)
--vigil-cpu None Max CPU usage (%)
--vigil-retry 0 Auto-retry failed/limit-violating tests
--vigil-stall-timeout None Max duration of low CPU (deadlock detection)
--vigil-session-timeout None Global timeout for entire test run
--vigil-json-report False Enable JSON reliability report saving
--vigil-output-dir .pytest_vigil Base directory for generated Vigil artifacts
--vigil-cli-report-verbosity short Terminal output: none, short, full
--vigil-force-exit-delay None Seconds after a soft interrupt before calling os._exit(124). Use when tests are stuck in GIL-holding C extensions. Disabled by default.

Using Markers

Apply specific limits to critical or heavy tests directly in code. All arguments are optional.

Parameter Type Unit Default Description
timeout float s None Test timeout
memory float MB None Memory limit
cpu float % None CPU limit
retry int - 0 Number of retries on failure
stall_timeout float s None Max duration of low CPU activity
stall_cpu_threshold float % 1.0 CPU threshold for stall detection
import pytest

@pytest.mark.vigil(timeout=5.0, memory=512, retry=2)
def test_heavy_computation():
    ...

Environment Variables

Perfect for CI/CD pipelines. All options are available via PYTEST_VIGIL__* prefix.

Variable Default Description
TIMEOUT None Default test timeout (seconds)
MEMORY_LIMIT_MB None Default memory limit (MB)
CPU_LIMIT_PERCENT None Default CPU limit (%)
SESSION_TIMEOUT None Global suite timeout (seconds)
SESSION_TIMEOUT_GRACE_PERIOD 5.0 Seconds to wait for graceful shutdown
MONITOR_INTERVAL 0.1 Internal check frequency (seconds)
STRICT_MODE True Enforce strict monitoring
CI_MULTIPLIER 2.0 Limit multiplier for CI environments
RETRY_COUNT 0 Number of retries for failures
STALL_TIMEOUT None Low-CPU deadlock timeout (seconds)
STALL_CPU_THRESHOLD 1.0 Threshold (%) for stall detection
CONSOLE_REPORT_VERBOSITY short Terminal output: none, short, full
JSON_REPORT_FILENAME vigil_report.json Default JSON report filename when reporting is enabled
JSON_REPORT False Enable JSON report saving by default
ARTIFACTS_DIR .pytest_vigil Base directory for generated Vigil artifacts
FORCE_EXIT_DELAY None Seconds after a soft interrupt before calling os._exit(124). Disabled by default.

📊 Reporting

Vigil provides insights into where your resources are going.

Terminal Report

Control verbosity with --vigil-cli-report-verbosity (none, short, full).

Short Mode (Default):

Vigil Reliability Report
Total Tests: 953 | Avg Duration: 5.32s | Avg Memory: 288.6 MB

Peak CPU by Process Type:
  Browser: 3542.1%  (Chromium/Webkit)
  Renderer: 2156.8% (Tab rendering)
  Pytest: 593.5%    (Test logic)

Full Mode:

Vigil Reliability Report
Test ID                                                 Att Duration (s)  Max CPU (%) Max Mem (MB)
--------------------------------------------------------------------------------------------------
tests/test_stress.py::test_high_load                      0         8.42        450.5        820.1
tests/test_ui.py::test_login[chromium]                    0         4.15       2101.2        415.8
tests/test_ui.py::test_checkout[chromium]                 1        12.30       3542.1        590.4
tests/test_api.py::test_latency                           0         0.25         15.2         45.1

💡 Note on CPU > 100%: In multi-process testing (like Playwright/Selenium), usage is summed across all cores and child processes. 7000% CPU usage means your test suite is utilizing ~70 cores efficiently (or inefficiently!).

JSON Report

The JSON report captures cpu_breakdown for every test, helping you identify if it's the Browser, DB, or Python code causing the spike.

Report saving is disabled by default. Enable it with --vigil-json-report; output goes to .pytest_vigil/vigil_report.json unless overridden.

Key Fields:

  • flaky_tests: Tests that passed after retry (attempt > 0)
  • cpu_breakdown: Peak CPU by process type (pytest, browser, renderer, gpu, webdriver, python, automation)
  • limits: Applied resource constraints from CLI/markers/env
📄 Example JSON Report (click to expand)
{
  "timestamp": "2026-02-08T14:23:45.123456+00:00",
  "flaky_tests": [
    "tests/test_integration.py::test_api_retry"
  ],
  "results": [
    {
      "node_id": "tests/test_ui.py::test_checkout[chromium]",
      "attempt": 0,
      "duration": 12.34,
      "max_cpu": 3542.1,
      "max_memory": 590.4,
      "cpu_breakdown": {
        "pytest": 89.2,
        "browser": 1805.3,
        "renderer": 1247.6,
        "gpu": 400.0
      },
      "limits": [
        {
          "limit_type": "time",
          "threshold": 15.0,
          "secondary_threshold": null,
          "strict": true
        },
        {
          "limit_type": "memory",
          "threshold": 1024.0,
          "secondary_threshold": null,
          "strict": true
        }
      ]
    },
    {
      "node_id": "tests/test_integration.py::test_api_retry",
      "attempt": 1,
      "duration": 2.15,
      "max_cpu": 45.8,
      "max_memory": 128.3,
      "cpu_breakdown": {
        "pytest": 45.8
      },
      "limits": [
        {
          "limit_type": "time",
          "threshold": 5.0,
          "secondary_threshold": null,
          "strict": true
        }
      ]
    }
  ]
}

⚖️ License

MIT

Metadata

Release files for pytest-vigil 0.8.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pytest-vigil 0.8.0
File Size Uploaded
pytest_vigil-0.8.0.tar.gz 19.8 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for pytest-vigil 0.8.0
File Interpreter ABI Platform
pytest_vigil-0.8.0-py3-none-any.whl Python 3 none any Details

Total release size: 49.0 kB

Release files / pytest_vigil-0.8.0.tar.gz

Download URL pytest_vigil-0.8.0.tar.gz
Size 19.8 kB
Tags Source
SHA-256 checksum
How to use checksums
559d1a38e2b3b739c47e8804da8bf94aee2cc7b66f5890d962c0d826d15fbc75
BLAKE2b-256 checksum
How to use checksums
923d5f18af684359e0a4835df30b1388bcd4213e399b1422fbfd94cdc7629c46
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Apr 7, 2026.

Transparency log

Release files / pytest_vigil-0.8.0-py3-none-any.whl

Download URL pytest_vigil-0.8.0-py3-none-any.whl
Size 29.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b4ddad93f68858521e1df60d0d5453d41a3c17c99db4d30b26a87f574a4e0f8a
BLAKE2b-256 checksum
How to use checksums
d2186716945349486daafb9ff29fc3298389f6396cb66765da1f60d87ee36fe0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Apr 7, 2026.

Transparency log

Release history Release notifications | RSS feed

This release

0.8.0 This release

2 release files

0.7.2

2 release files

0.7.1

2 release files

0.7.0

2 release files

0.6.2

2 release files

0.6.1

2 release files

0.6.0

2 release files

0.5.1

2 release files

0.5.0

2 release files

0.4.0

2 release files

0.3.0

2 release files

0.2.0

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page