testrelic-playwright

A pytest + Playwright analytics reporter that generates structured JSON and HTML reports from your Playwright test runs — capturing navigation timelines, API call tracking, network statistics, failure diagnostics, and CI metadata. The Python port of @testrelic/playwright-analytics.

What it does

When you run your tests with the TestRelic pytest plugin you get a JSON + HTML report that captures:

• Navigation timeline — every URL visited during each test in chronological order, with navigation type detection (goto, link click, back/forward, SPA route changes, hash changes).
• API call tracking — HTTP method, URL, status code, request/response headers and bodies, response times, and assertions for every API call.
• Network statistics — total requests, failed requests, bytes transferred, and resource-type breakdowns (scripts, images, stylesheets, fonts, XHR).
• Test results — pass/fail/flaky status, duration, retry count, and tags.
• Failure diagnostics — error messages, source-code snippets pointing to the exact failure line, and optional stack traces.
• CI metadata — auto-detection of GitHub Actions, GitLab CI, Jenkins, and CircleCI with build id, commit sha, and branch.
• Sensitive-data redaction — AWS keys, Bearer tokens, private keys, credential URLs, and sensitive API headers / body fields.
• Action steps — page actions (goto, click, fill, …) and expect assertions captured as a categorised, video-synced step timeline.
• Assertion tracking — every expect() recorded with pass/fail, expected / actual, source location, and linkage to the API call it checked.
• Console log capture — browser console messages (log / warn / error) per test.
• Screenshots & videos — captured per test into an artifacts/ folder alongside the report and embedded in the HTML.
• Interactive HTML report — single-file SPA with a test grid, search + status/type/file filters, a per-test drawer, a network DevTools panel, console and action-step panels, an artifact/video viewer, and a failure panel with a "Copy Prompt" button for AI-assisted repair.
• Large-suite streaming mode — for big suites the report streams to disk and is served locally (testrelic-playwright serve …).
• Terminal summary — a box-drawing run summary printed at the end, with the report auto-opened in your browser (outside CI).

Quick start

1. Install

pip install pytest-playwright testrelic-playwright
playwright install

The reporter activates automatically because testrelic-playwright registers a pytest11 entry point. No conftest.py plumbing required.

2. Write a test

Use the standard page and context fixtures from pytest-playwright. The TestRelic plugin auto-attaches listeners and records navigation / API calls.

# tests/test_homepage.py
from playwright.sync_api import Page, expect


def test_homepage_loads(page: Page) -> None:
    page.goto("https://example.com")
    expect(page.locator("h1")).to_be_visible()

API tests work the same way using pytest-playwright's built-in APIRequestContext fixture:

from playwright.sync_api import APIRequestContext


def test_fetch_posts(playwright) -> None:
    request = playwright.request.new_context()
    response = request.get("https://api.example.com/posts")
    assert response.status == 200

3. Run

pytest

Outputs land in:

test-results/
├── analytics-timeline.json
└── analytics-timeline.html

Configuration

Reporter options can be supplied three ways (highest priority first):

1. Environment variables (TESTRELIC_API_KEY, TESTRELIC_CLOUD_ENDPOINT, TESTRELIC_UPLOAD_STRATEGY, TESTRELIC_CLOUD_TIMEOUT).
2. pytest.ini / pyproject.toml testrelic_options ini block.
3. .testrelic/testrelic-config.json in the project root.

pytest.ini

[pytest]
testrelic_options =
    outputPath=./reports/run.json
    includeStackTrace=true
    captureRequestBody=true
    captureResponseBody=true

pyproject.toml

[tool.pytest.ini_options]
testrelic_options = [
  "outputPath=./reports/run.json",
  "includeStackTrace=true",
]

CLI flags

pytest --testrelic-output ./reports/custom.json
pytest --testrelic-disable    # turn the reporter off for this run
pytest --testrelic-quiet      # suppress banner output

Full option list

Cloud integration

Set TESTRELIC_API_KEY and the reporter uploads the batch payload to https://platform.testrelic.ai/api/v1/test-runs at session end.

export TESTRELIC_API_KEY=tk_live_...
export TESTRELIC_UPLOAD_STRATEGY=both   # realtime | batch | both

Or use a project file:

// .testrelic/testrelic-config.json
{
  "cloud": {
    "apiKey": "$TESTRELIC_API_KEY",
    "endpoint": "https://platform.testrelic.ai/api/v1",
    "upload": "both"
  },
  "testrelic-repo": {
    "name": "my-project"
  }
}

Network failures are queued to .testrelic/queue/ and can be retried with:

testrelic-playwright drain

CLI

testrelic-playwright merge shard-1.json shard-2.json -o merged.json
testrelic-playwright serve ./test-results/.testrelic-report --port 9323
testrelic-playwright open  ./test-results/.testrelic-report   # serve + open browser
testrelic-playwright drain
testrelic-playwright version

serve runs a real local report server (/api/* routes, automatic port selection from 9323, 30-minute inactivity shutdown) for streamed large-suite reports; for a single embedded HTML report it serves the file directly.

Manual navigation hook

For navigation the auto-tracker can't see (custom routing, iframes), use record_navigation:

from testrelic_playwright import record_navigation


def test_with_manual_nav(page, testrelic_state) -> None:
    page.goto("https://example.com")
    record_navigation(testrelic_state, "https://example.com/spa/profile", "spa_route")

Compatibility

• Python 3.9 – 3.12
• pytest 7.0+
• pytest-playwright 0.5+
• Playwright 1.35+

License

MIT