pytest-beacon 0.3.0

Highly customizable pytest reporting plugin combining AI-optimized CTRF reports and rich test metrics

Pytest Beacon

Pytest Beacon is a highly customizable pytest reporting plugin. Generates CTRF 1.0.0 reports with rich test metadata — marks, parameters, Allure IDs, environment — and exports them to a local file and/or a remote HTTP service.

Why you might need this

• You want machine-readable test results that CI/CD pipelines can consume
• Your test suite is large and you need token-efficient reports (passed tests excluded by default)
• You send test metrics to a remote service or dashboard for trend analysis
• You run tests in parallel with pytest-xdist and need a unified report
• You need collection errors (import errors, syntax errors) tracked alongside test results

---

✨ Features

• CTRF 1.0.0 compliance: JSON/YAML reports following the Common Test Results Format standard
• Token-efficient by default: Passed tests excluded from output; summary counters are always accurate
• Independent output filtering: Separate exclude rules for local file and HTTP export
• Rich metadata: Marks, parametrized params, Allure IDs, and environment name per test
• HTTP export: POST reports to a remote metrics service after each run
• xdist support: Full pytest-xdist compatibility for parallel execution
• Collection error tracking: Import errors and syntax errors appear in the report
• Flexible configuration: CLI flags, environment variables, and .env file support

🚀 Installation

uv add pytest-beacon
# or
pip install pytest-beacon

⚡ Quick Start

1. Generate a local report Writes to beacon_reports/report-YYYYMMDD-HHMMSS.json by default:

pytest --beacon

2. Custom file and format

pytest --beacon --beacon-file=results.json --beacon-format=yaml

3. Send to a remote service

pytest --beacon --beacon-url=http://localhost:8000/api/v1/metrics

4. Include all statuses and captured output

pytest --beacon --beacon-file-exclude-status='' --beacon-verbose

7. Send all statuses to HTTP but keep local file lean

pytest --beacon --beacon-url=http://localhost:8000/api/v1/metrics --beacon-http-exclude-status=''

5. Parallel execution with xdist

pytest --beacon -n auto

6. Attach arbitrary metadata

pytest --beacon --beacon-meta build=123 --beacon-meta branch=main --beacon-meta triggered_by=ci

🛠 CLI Options

Option	Default	Description
--beacon	off	Enable reporting
--beacon-file PATH	beacon_reports/report-<ts>.json	Output file. Bare filename → placed in beacon_reports/ with timestamp.
--beacon-url URL	—	Full URL to POST the report to.
--beacon-format json|yaml	json	Report serialisation format
--beacon-verbose	off	Include captured stdout/stderr for passed tests
--beacon-file-exclude-status STATUSES	passed	Comma-separated statuses to omit from the local file report. Empty string includes all.
--beacon-http-exclude-status STATUSES	passed	Comma-separated statuses to omit from the HTTP export. Empty string includes all.
--beacon-meta KEY=VALUE	—	Arbitrary metadata pair added to the report environment. Repeatable.

Environment Variables

All variables use the PYTEST_BEACON__ prefix. Can also be set in a .env file at the project root.

Variable	Default	Description
PYTEST_BEACON__REPORT_FORMAT	json	Default report format
PYTEST_BEACON__FILE_EXCLUDE_STATUSES	passed	Statuses to omit from local file report
PYTEST_BEACON__HTTP_EXCLUDE_STATUSES	passed	Statuses to omit from HTTP export
PYTEST_BEACON__HTTP_TIMEOUT	10.0	HTTP export timeout in seconds
PYTEST_BEACON__HTTP_MAX_RETRIES	3	HTTP export retry attempts

📊 Report Format

Summary counters always reflect all tests regardless of --beacon-file-exclude-status or --beacon-http-exclude-status.

Beacon also stores a pytest-style summary in results.extra.pytestSummary when pytest's terminal reporter is available. This mirrors pytest's own terminal categories as closely as possible, which makes it easy to compare beacon output with the pytest final line.

The local console output includes this as an extra line, for example:

Pytest:  79 failed, 759 passed, 123 skipped, 3 xfailed, 57 warnings, 12 errors, 4 rerun in 1211.04s

📄 Example CTRF JSON Report (click to expand)

{
  "results": {
    "tool": { "name": "pytest", "version": "9.x" },
    "summary": {
      "tests": 12, "passed": 9, "failed": 2, "skipped": 1, "error": 0,
      "start": 1700000000000, "stop": 1700000005000
    },
    "tests": [
      {
        "name": "tests/test_api.py::test_login[admin]",
        "status": "failed",
        "duration": 142,
        "filePath": "tests/test_api.py",
        "line": 34,
        "message": "AssertionError: expected 200, got 401",
        "trace": "...",
        "marks": ["smoke", "auth"],
        "params": { "role": "admin" },
        "allureId": "TC-42"
      }
    ],
    "environment": { "pythonVersion": "3.12.0", "pytestVersion": "9.0.0", "xdistWorkers": 4, "build": "123", "branch": "main" },
    "extra": {
      "pluginName": "pytest-beacon",
      "pluginVersion": "0.2.0",
      "ctrf": "1.0.0",
      "pytestSummary": {
        "failed": 2, "passed": 9, "skipped": 1, "deselected": 0,
        "xfailed": 0, "xpassed": 0, "warnings": 0, "error": 0, "rerun": 0
      }
    }
  }
}

📤 Remote HTTP Export

When --beacon-url is set, a POST request with a JSON body is sent to the specified URL after the run. Export errors are logged and never interrupt test execution.

📄 Example HTTP Request Body (click to expand)

{
  "metrics": [
    {
      "test_nodeid": "tests/test_api.py::test_login[admin]",
      "test_name": "test_login[admin]",
      "test_result": "failed",
      "test_duration": 0.142,
      "test_marks": ["smoke", "auth"],
      "test_params": { "role": "admin" },
      "test_stacktrace": "...",
      "test_message": "AssertionError: expected 200, got 401",
      "test_allure_id": "TC-42"
    }
  ],
  "environment": {
    "pythonVersion": "3.12.0",
    "pytestVersion": "9.0.0",
    "xdistWorkers": 4,
    "build": "123",
    "branch": "main"
  }
}

Local Development

Install dependencies:

uv sync

Run tests:

uv run pytest tests/

Run a quick smoke test against your own test suite:

uv run pytest your_tests/ --beacon --beacon-file-exclude-status=

⚖️ License

MIT