Skip to main content

pytest-vigil

Pytest Vigil

License: MIT Python GitHub last commit GitHub Release Date

Pytest Vigil is a reliability pytest plugin that enforces resource limits on your tests and kills them when they exceed those limits.

Why you might need this

  • Tests sometimes hang indefinitely due to deadlocks or infinite loops
  • Memory leaks crash your test runner or CI environment
  • CPU-intensive tests slow down your entire suite
  • You want to enforce maximum runtime for your CI pipeline
  • You need to identify which tests are resource hogs

✨ Features

  • Deadlock detection: Kill tests that hang with low CPU activity
  • Suite timeout: Stop the entire test run after a specified duration
  • CI scaling: Automatically relaxes limits by 2x in CI environments (configurable)
  • Retry mechanism: Re-run tests that fail due to resource violations
  • Detailed reports: JSON output showing CPU breakdown by process type (browser, renderer, etc.)
  • Per-test timeout: Layered timeout enforcement — kernel-level alarm backstop, C-level faulthandler diagnostics, and optional force-exit escalation for tests stuck in GIL-holding C extensions
  • Per-test memory limit: Kills tests that exceed memory limits
  • Per-test CPU limit: Kills tests that exceed CPU usage limits

🚀 Installation

uv add -D pytest-vigil
# or
pip install pytest-vigil

⚡ Quick Start

1. Protect against heavy tests Limit tests to 5 seconds, 512MB RAM, and 80% CPU:

pytest --vigil-timeout 5 --vigil-memory 512 --vigil-cpu 80

2. Prevent infinite CI hangs Kill the entire suite if it runs longer than 15 minutes:

pytest --vigil-session-timeout 900

3. Generate Reliability Report

pytest --vigil-json-report
# saved to .pytest_vigil/vigil_report.json

🛠 Usage & Configuration

CLI Options Reference

Option Default Description
--vigil-timeout None Max duration per test (seconds)
--vigil-memory None Max memory usage (MB)
--vigil-cpu None Max CPU usage (%)
--vigil-retry 0 Auto-retry failed/limit-violating tests
--vigil-stall-timeout None Max duration of low CPU (deadlock detection)
--vigil-session-timeout None Global timeout for entire test run
--vigil-json-report False Enable JSON reliability report saving
--vigil-output-dir .pytest_vigil Base directory for generated Vigil artifacts
--vigil-cli-report-verbosity short Terminal output: none, short, full
--vigil-force-exit-delay None Seconds after a soft interrupt before calling os._exit(124). Use when tests are stuck in GIL-holding C extensions. Disabled by default.

Using Markers

Apply specific limits to critical or heavy tests directly in code. All arguments are optional.

Parameter Type Unit Default Description
timeout float s None Test timeout
memory float MB None Memory limit
cpu float % None CPU limit
retry int - 0 Number of retries on failure
stall_timeout float s None Max duration of low CPU activity
stall_cpu_threshold float % 1.0 CPU threshold for stall detection
import pytest

@pytest.mark.vigil(timeout=5.0, memory=512, retry=2)
def test_heavy_computation():
    ...

Environment Variables

Perfect for CI/CD pipelines. All options are available via PYTEST_VIGIL__* prefix.

Variable Default Description
TIMEOUT None Default test timeout (seconds)
MEMORY_LIMIT_MB None Default memory limit (MB)
CPU_LIMIT_PERCENT None Default CPU limit (%)
SESSION_TIMEOUT None Global suite timeout (seconds)
SESSION_TIMEOUT_GRACE_PERIOD 5.0 Seconds to wait for graceful shutdown
MONITOR_INTERVAL 0.1 Internal check frequency (seconds)
STRICT_MODE True Enforce strict monitoring
CI_MULTIPLIER 2.0 Limit multiplier for CI environments
RETRY_COUNT 0 Number of retries for failures
STALL_TIMEOUT None Low-CPU deadlock timeout (seconds)
STALL_CPU_THRESHOLD 1.0 Threshold (%) for stall detection
CONSOLE_REPORT_VERBOSITY short Terminal output: none, short, full
JSON_REPORT_FILENAME vigil_report.json Default JSON report filename when reporting is enabled
JSON_REPORT False Enable JSON report saving by default
ARTIFACTS_DIR .pytest_vigil Base directory for generated Vigil artifacts
FORCE_EXIT_DELAY None Seconds after a soft interrupt before calling os._exit(124). Disabled by default.

📊 Reporting

Vigil provides insights into where your resources are going.

Terminal Report

Control verbosity with --vigil-cli-report-verbosity (none, short, full).

Short Mode (Default):

Vigil Reliability Report
Total Tests: 953 | Avg Duration: 5.32s | Avg Memory: 288.6 MB

Peak CPU by Process Type:
  Browser: 3542.1%  (Chromium/Webkit)
  Renderer: 2156.8% (Tab rendering)
  Pytest: 593.5%    (Test logic)

Full Mode:

Vigil Reliability Report
Test ID                                                 Att Duration (s)  Max CPU (%) Max Mem (MB)
--------------------------------------------------------------------------------------------------
tests/test_stress.py::test_high_load                      0         8.42        450.5        820.1
tests/test_ui.py::test_login[chromium]                    0         4.15       2101.2        415.8
tests/test_ui.py::test_checkout[chromium]                 1        12.30       3542.1        590.4
tests/test_api.py::test_latency                           0         0.25         15.2         45.1

💡 Note on CPU > 100%: In multi-process testing (like Playwright/Selenium), usage is summed across all cores and child processes. 7000% CPU usage means your test suite is utilizing ~70 cores efficiently (or inefficiently!).

JSON Report

The JSON report captures cpu_breakdown for every test, helping you identify if it's the Browser, DB, or Python code causing the spike.

Report saving is disabled by default. Enable it with --vigil-json-report; output goes to .pytest_vigil/vigil_report.json unless overridden.

Key Fields:

  • flaky_tests: Tests that passed after retry (attempt > 0)
  • cpu_breakdown: Peak CPU by process type (pytest, browser, renderer, gpu, webdriver, python, automation)
  • limits: Applied resource constraints from CLI/markers/env
📄 Example JSON Report (click to expand)
{
  "timestamp": "2026-02-08T14:23:45.123456+00:00",
  "flaky_tests": [
    "tests/test_integration.py::test_api_retry"
  ],
  "results": [
    {
      "node_id": "tests/test_ui.py::test_checkout[chromium]",
      "attempt": 0,
      "duration": 12.34,
      "max_cpu": 3542.1,
      "max_memory": 590.4,
      "cpu_breakdown": {
        "pytest": 89.2,
        "browser": 1805.3,
        "renderer": 1247.6,
        "gpu": 400.0
      },
      "limits": [
        {
          "limit_type": "time",
          "threshold": 15.0,
          "secondary_threshold": null,
          "strict": true
        },
        {
          "limit_type": "memory",
          "threshold": 1024.0,
          "secondary_threshold": null,
          "strict": true
        }
      ]
    },
    {
      "node_id": "tests/test_integration.py::test_api_retry",
      "attempt": 1,
      "duration": 2.15,
      "max_cpu": 45.8,
      "max_memory": 128.3,
      "cpu_breakdown": {
        "pytest": 45.8
      },
      "limits": [
        {
          "limit_type": "time",
          "threshold": 5.0,
          "secondary_threshold": null,
          "strict": true
        }
      ]
    }
  ]
}

⚖️ License

MIT

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest_vigil-0.8.0.tar.gz (19.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pytest_vigil-0.8.0-py3-none-any.whl (29.2 kB view details)

Uploaded Python 3

File details

Details for the file pytest_vigil-0.8.0.tar.gz.

File metadata

  • Download URL: pytest_vigil-0.8.0.tar.gz
  • Upload date:
  • Size: 19.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for pytest_vigil-0.8.0.tar.gz
Algorithm Hash digest
SHA256 559d1a38e2b3b739c47e8804da8bf94aee2cc7b66f5890d962c0d826d15fbc75
MD5 abc225d90a7039fb28d4a5905b976ec8
BLAKE2b-256 923d5f18af684359e0a4835df30b1388bcd4213e399b1422fbfd94cdc7629c46

See more details on using hashes here.

Provenance

The following attestation bundles were made for pytest_vigil-0.8.0.tar.gz:

Publisher: publish-to-pypi.yml on l0kifs/pytest-vigil

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pytest_vigil-0.8.0-py3-none-any.whl.

File metadata

  • Download URL: pytest_vigil-0.8.0-py3-none-any.whl
  • Upload date:
  • Size: 29.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for pytest_vigil-0.8.0-py3-none-any.whl
Algorithm Hash digest
SHA256 b4ddad93f68858521e1df60d0d5453d41a3c17c99db4d30b26a87f574a4e0f8a
MD5 b2ddb54c496f5320c9836757039a812c
BLAKE2b-256 d2186716945349486daafb9ff29fc3298389f6396cb66765da1f60d87ee36fe0

See more details on using hashes here.

Provenance

The following attestation bundles were made for pytest_vigil-0.8.0-py3-none-any.whl:

Publisher: publish-to-pypi.yml on l0kifs/pytest-vigil

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.8.0 This release

2 files

0.7.2

2 files

0.7.1

2 files

0.7.0

2 files

0.6.2

2 files

0.6.1

2 files

0.6.0

2 files

0.5.1

2 files

0.5.0

2 files

0.4.0

2 files

0.3.0

2 files

0.2.0

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page