pytest-smart-rerun 0.1.3

A Pytest plugin for intelligent retrying of flaky tests.

pytest-smart-rerun

pytest-smart-rerun is a lightweight Pytest plugin that retries flaky tests with configurable attempts and delays. It integrates directly with Pytest's hook system, requires no external dependencies, and works out of the box from the command line or via per-test decorators.

Installation

pip install pytest-smart-rerun

For local development:

git clone https://github.com/Aki-07/pytest-smart-rerun.git
cd pytest-smart-rerun
pip install -e .

Quick start

Retry all failing tests up to three times with a one second delay:

pytest --smart-rerun --smart-rerun-max=3 --smart-rerun-delay=1

Use the smart_rerun marker to target specific flaky tests:

import pytest


@pytest.mark.smart_rerun(max=4, delay=0.5)
def test_eventually_consistent_service(client):
    response = client.read_state()
    assert response.status_code == 200

Combine both for a default retry policy with test-level overrides.

CLI options

• --smart-rerun: enable smart reruns for the test session.
• --smart-rerun-max=N: set the total number of attempts per test (default: 2).
• --smart-rerun-delay=SECONDS: add a delay between retries (default: 0).
• --smart-rerun-backoff=linear|exp|none: apply linear or exponential backoff to delay intervals.
• --smart-rerun-errors=name1,name2: only retry failures whose exception names match the allow-list.
• --smart-rerun-report=PATH: write structured JSON analytics for downstream dashboards or triage tooling.
• --smart-rerun-ai: enable MCP-powered AI analysis to classify failures and adapt retry behaviour.
• --smart-rerun-ai-report=PATH: append AI classifications and suggested actions to a JSON artifact.

Invalid values raise a Pytest UsageError, keeping failure feedback obvious.

Configuration via pytest.ini

All CLI options have config equivalents so you can bake defaults into your repository:

[pytest]
smart_rerun = true
smart_rerun_max = 3
smart_rerun_delay = 0.5
smart_rerun_backoff = exp
smart_rerun_errors = TimeoutError,ConnectionError
smart_rerun_report = reports/smart_rerun.json
smart_rerun_ai = true
smart_rerun_ai_endpoint = https://mcp.example.com/analyze
smart_rerun_ai_token = {{ env:SMART_RERUN_AI_TOKEN }}
smart_rerun_ai_report = reports/smart_rerun_ai.json
smart_rerun_ai_cache = .cache/smart_rerun_ai.json
smart_rerun_ai_redact = true

Features

• Intelligent reruns driven purely by Pytest hooks—no external dependencies.
• Automatic attempt logging with rerun-aware, colourised terminal summaries ready for CI.
• Per-test overrides through @pytest.mark.smart_rerun(max=..., delay=..., backoff=..., errors=...).
• Linear or exponential backoff strategies that cooperate with scripted delays.
• Exception-aware filtering so only flaky categories (e.g. timeouts) trigger reruns.
• Structured JSON analytics for dashboards or historical insights via --smart-rerun-report.
• AI-assisted classification that adjusts retry strategy on the fly (delay overrides, early exits, rich reporting).
• Works alongside existing Pytest plugins and respects fixture/state isolation.

AI-powered smart reruns

Turn on the MCP-backed engine with either --smart-rerun-ai or smart_rerun_ai = true. When a test fails, the plugin gathers the stack trace, exception metadata, and retry configuration, then:

• Sends it to the configured MCP endpoint (or falls back to built-in heuristics when offline).
• Classifies the failure (flaky-network, deterministic, environmental, …) with a confidence score.
• Adapts the retry policy dynamically—e.g. retry_with_delay(2s) for network blips or stop_retry for logic bugs.
• Writes rich analytics into the main JSON report and, if configured, an AI-specific trail (--smart-rerun-ai-report).

Caching (--smart-rerun-ai-cache) avoids duplicate calls, --smart-rerun-ai-redact trims overly verbose traces before transmission, and environment variables such as SMART_RERUN_AI_TOKEN can feed credentials securely.

CLI analytics

Run smart-rerun-report path/to/report.json (or multiple files) to see an aggregated dashboard in your terminal. Pass --format=json for machine-readable output or wire it into CI artefacts:

pytest --smart-rerun --smart-rerun-report reports/smart.json
smart-rerun-report reports/smart.json
smart-rerun-report reports/smart.json reports/previous.json --format=json

Future roadmap

1. Natural-language retry policies that the MCP agent translates into structured plugin config.
2. Correlating repeated failures across repositories to suggest code-level fixes automatically.
3. Optional dashboards that visualise the JSON analytics over time for QA and release teams.

Development

python -m venv .venv
source .venv/bin/activate
pip install -U pip setuptools wheel
pip install -e .[test]
pytest -v
pytest --smart-rerun --smart-rerun-max=3 --smart-rerun-delay=1 -v

Release workflow

1. python -m build
2. python -m twine check dist/*
3. python -m twine upload dist/*

Refer to Publishing to PyPI for account setup and API tokens.

License

MIT © 2025 Akilesh KR. See LICENSE for full text.