testrelic-pytest

Generic pytest reporter that ships your test results to the TestRelic AI cloud platform. Works with any pytest project — unit tests, API tests, integration tests — and stays out of the way when no API key is configured.

Install

pip install testrelic-pytest

The plugin auto-registers via the pytest11 entry point. Nothing to import.

Configure

export TESTRELIC_API_KEY=tr_live_...
pytest

The plugin silently no-ops when TESTRELIC_API_KEY is unset, so it is safe to add to any project.

Options

All options can be set via CLI flag, environment variable, or a JSON file at .testrelic/testrelic-config.json. Precedence (highest first): CLI flag → environment → JSON file → defaults.

CLI flags use a --testrelic-pytest-* prefix so they don't collide with the testrelic-playwright plugin if both are installed.

Config file (.testrelic/testrelic-config.json)

{
  "apiKey": "$TESTRELIC_API_KEY",
  "endpoint": "https://platform.testrelic.ai/api/v1",
  "upload": "batch",
  "projectName": "my-suite",
  "artifactThresholdKb": 32,
  "uploadArtifacts": true,
  "queueDirectory": ".testrelic/queue",
  "metadata": { "team": "platform" }
}

$ENV_VAR strings are expanded against the process environment at startup.

What gets captured

For each test the plugin captures:

• nodeId, file, className, testName, parametrizeId
• Outcome: passed | failed | skipped | xfailed | xpassed | error
• Duration (ms) and per-phase breakdown (setup/call/teardown)
• Markers (e.g. @pytest.mark.smoke), keywords, parametrize ids
• Skip reasons and pytest warnings
• Failure tracebacks and assertion repr
• Captured stdout / stderr / caplog (inline with redaction; truncated above --testrelic-artifact-threshold-kb, hard cap 1 MB per stream — artifact upload for large streams is planned for v0.2)

Test runs include CI metadata auto-detected from GitHub Actions, GitLab CI, Jenkins, and CircleCI, plus git branch/commit/author from the working tree.

pytest-xdist

Parallel runs via pytest-xdist (pytest -n 4) are supported out of the box. Each worker forwards its TestReport objects through the xdist channel; only the controller node uploads. The result is exactly one cloud run with all tests aggregated.

Offline mode

Any upload that fails after retries is written to ~/.testrelic/queue/ as a JSON file. Re-run later:

testrelic-pytest drain

The drainer re-authenticates with TESTRELIC_API_KEY and replays each entry to its original endpoint.

See also

• testrelic-deepeval — DeepEval / LLM evaluation bridge
• testrelic-playwright — Playwright reporter for pytest