testrelic-playwright 0.3.4

Playwright/pytest test analytics reporter — navigation timeline, API call capture, network stats, failure diagnostics, and interactive HTML reports.

testrelic-playwright

A pytest + Playwright analytics reporter that generates structured JSON and HTML reports from your Playwright test runs — capturing navigation timelines, API call tracking, network statistics, failure diagnostics, and CI metadata. The Python port of @testrelic/playwright-analytics.

What it does

When you run your tests with the TestRelic pytest plugin you get a JSON + HTML report that captures:

• Navigation timeline — every URL visited during each test in chronological order, with navigation type detection (goto, link click, back/forward, SPA route changes, hash changes).
• API call tracking — HTTP method, URL, status code, request/response headers and bodies, response times, and assertions for every API call.
• Network statistics — total requests, failed requests, bytes transferred, and resource-type breakdowns (scripts, images, stylesheets, fonts, XHR).
• Test results — pass/fail/flaky status, duration, retry count, and tags.
• Failure diagnostics — error messages, source-code snippets pointing to the exact failure line, and optional stack traces.
• CI metadata — auto-detection of GitHub Actions, GitLab CI, Jenkins, and CircleCI with build id, commit sha, and branch.
• Sensitive-data redaction — AWS keys, Bearer tokens, private keys, credential URLs, and sensitive API headers / body fields.
• Action steps — page actions (goto, click, fill, …) and expect assertions captured as a categorised, video-synced step timeline.
• Assertion tracking — every expect() recorded with pass/fail, expected / actual, source location, and linkage to the API call it checked.
• Console log capture — browser console messages (log / warn / error) per test.
• Screenshots & videos — captured per test into an artifacts/ folder alongside the report and embedded in the HTML.
• Interactive HTML report — single-file SPA with a test grid, search + status/type/file filters, a per-test drawer, a network DevTools panel, console and action-step panels, an artifact/video viewer, and a failure panel with a "Copy Prompt" button for AI-assisted repair.
• Large-suite streaming mode — for big suites the report streams to disk and is served locally (testrelic-playwright serve …).
• Terminal summary — a box-drawing run summary printed at the end, with the report auto-opened in your browser (outside CI).

Quick start

1. Install

pip install pytest-playwright testrelic-playwright
playwright install

The reporter activates automatically because testrelic-playwright registers a pytest11 entry point. No conftest.py plumbing required.

2. Write a test

Use the standard page and context fixtures from pytest-playwright. The TestRelic plugin auto-attaches listeners and records navigation / API calls.

# tests/test_homepage.py
from playwright.sync_api import Page, expect


def test_homepage_loads(page: Page) -> None:
    page.goto("https://example.com")
    expect(page.locator("h1")).to_be_visible()

API tests work the same way using pytest-playwright's built-in APIRequestContext fixture:

from playwright.sync_api import APIRequestContext


def test_fetch_posts(playwright) -> None:
    request = playwright.request.new_context()
    response = request.get("https://api.example.com/posts")
    assert response.status == 200

3. Run

pytest

Outputs land in:

test-results/
├── analytics-timeline.json
└── analytics-timeline.html

Configuration

Reporter options can be supplied three ways (highest priority first):

1. Environment variables (TESTRELIC_API_KEY, TESTRELIC_CLOUD_ENDPOINT, TESTRELIC_UPLOAD_STRATEGY, TESTRELIC_CLOUD_TIMEOUT).
2. pytest.ini / pyproject.toml testrelic_options ini block.
3. .testrelic/testrelic-config.json in the project root.

pytest.ini

[pytest]
testrelic_options =
    outputPath=./reports/run.json
    includeStackTrace=true
    captureRequestBody=true
    captureResponseBody=true

pyproject.toml

[tool.pytest.ini_options]
testrelic_options = [
  "outputPath=./reports/run.json",
  "includeStackTrace=true",
]

CLI flags

pytest --testrelic-output ./reports/custom.json
pytest --testrelic-disable    # turn the reporter off for this run
pytest --testrelic-quiet      # suppress banner output

Full option list

Option	Type	Default	Description
outputPath	str	./test-results/analytics-timeline.json	JSON report path
htmlReportPath	str	derived from outputPath	HTML report path
includeStackTrace	bool	false	Include full stack traces in failures
includeCodeSnippets	bool	true	Include source code around the failure line
codeContextLines	int	3	Lines of context above / below the failure line
includeNetworkStats	bool	true	Track network requests and bytes per navigation
includeArtifacts	bool	true	Include screenshots and videos
includeActionSteps	bool	true	Capture pytest test steps
captureConsoleLogs	bool	true	Capture browser console messages
trackApiCalls	bool	true	Enable / disable API call tracking
captureRequestHeaders	bool	true	Capture request headers
captureResponseHeaders	bool	true	Capture response headers
captureRequestBody	bool	true	Capture request bodies
captureResponseBody	bool	true	Capture response bodies
redactHeaders	list[str]	[authorization, cookie, set-cookie, x-api-key]	Header names to redact
redactBodyFields	list[str]	[password, secret, token, apiKey, api_key]	JSON body field names to scrub
apiIncludeUrls	list[str]	[]	Only track API calls matching these patterns (glob/regex)
apiExcludeUrls	list[str]	[]	Exclude API calls matching these patterns (takes precedence)
captureAssertions	bool	true	Capture expect() assertion results
openReport	bool	true	Auto-open the HTML report after the run (skipped in CI)
reportMode	str	auto	embedded / streaming / auto (streams large suites to disk)
streamingThreshold	int	500	Test count at which auto switches to streaming
quiet	bool	false	Suppress banner + terminal summary output

Cloud integration

Set TESTRELIC_API_KEY and the reporter uploads the batch payload to https://platform.testrelic.ai/api/v1/test-runs at session end.

export TESTRELIC_API_KEY=tk_live_...
export TESTRELIC_UPLOAD_STRATEGY=both   # realtime | batch | both

Or use a project file:

// .testrelic/testrelic-config.json
{
  "cloud": {
    "apiKey": "$TESTRELIC_API_KEY",
    "endpoint": "https://platform.testrelic.ai/api/v1",
    "upload": "both"
  },
  "testrelic-repo": {
    "name": "my-project"
  }
}

Network failures are queued to .testrelic/queue/ and can be retried with:

testrelic-playwright drain

CLI

testrelic-playwright merge shard-1.json shard-2.json -o merged.json
testrelic-playwright serve ./test-results/.testrelic-report --port 9323
testrelic-playwright open  ./test-results/.testrelic-report   # serve + open browser
testrelic-playwright drain
testrelic-playwright version

serve runs a real local report server (/api/* routes, automatic port selection from 9323, 30-minute inactivity shutdown) for streamed large-suite reports; for a single embedded HTML report it serves the file directly.

Manual navigation hook

For navigation the auto-tracker can't see (custom routing, iframes), use record_navigation:

from testrelic_playwright import record_navigation


def test_with_manual_nav(page, testrelic_state) -> None:
    page.goto("https://example.com")
    record_navigation(testrelic_state, "https://example.com/spa/profile", "spa_route")

Compatibility

• Python 3.9 – 3.12
• pytest 7.0+
• pytest-playwright 0.5+
• Playwright 1.35+

License

MIT