testrelic-appium

Appium test analytics reporter for pytest — captures command timelines, device logs (Android logcat / iOS syslog), console logs, network requests, screenshots, and video, then generates a self-contained interactive HTML report. Python port of @testrelic/appium-analytics.

What it does

When you run your Appium tests with pytest and the testrelic-appium plugin installed, you get JSON + HTML reports that capture:

• Command timeline — every Appium / WebDriver command with timing, category (find, interaction, gesture, navigation, lifecycle, context, device, keyboard, alert, screenshot, other), selector, arguments (sensitive ones redacted), and result.
• Device logs — Android logcat, iOS syslog and crashlog, polled in the background and sliced per test.
• Console logs — WebView / browser console output via WebDriver BiDi (log.entryAdded) when supported, with polling fallback (browser, safariConsole).
• Network capture — Chrome DevTools Protocol performance logs (Android), safariNetwork (iOS), with optional mitmproxy fallback for emulators without CDP.
• Screenshots & video — configurable: off, on every test, on failure, or on every step. Video uses Appium recording commands first, falling back to adb screenrecord.
• Assertions — assert_that(...) context manager records named assertion steps; bare assert statements are picked up by pytest_assertion_pass.
• CI metadata — auto-detect GitHub Actions, GitLab CI, Jenkins, CircleCI, Bitbucket Pipelines.
• Sensitive-data redaction — AWS access keys, Bearer tokens, PEM private keys, embedded URL credentials, plus Appium-specific command-argument redaction (elementSendKeys, setValueImmediate, replaceValue).
• HTML report — self-contained, opens in any browser.

System requirements

The package itself has no platform requirements beyond Python, but to test mobile apps you'll need the standard Appium tooling on your machine. None of these are pip dependencies — they are subprocess dependencies the SDK invokes when needed.

iOS testing requires macOS 13+ with Xcode 15+. iOS tests cannot run on Windows or Linux.

Quick start

1. Install

pip install testrelic-appium

The plugin activates automatically because testrelic-appium registers a pytest11 entry point. No conftest.py plumbing required.

For the optional HTTPS network-capture proxy:

pip install "testrelic-appium[network-proxy]"

2. Write a test

Use any Appium driver instantiation pattern. The plugin walks item.funcargs after each fixture resolves and per-instance wraps any appium.webdriver.webdriver.WebDriver it finds — Selenium-only tests are never touched.

# tests/test_login.py
import pytest
from appium import webdriver
from appium.options.android import UiAutomator2Options


@pytest.fixture
def driver():
    options = UiAutomator2Options()
    options.platform_name = "Android"
    options.device_name = "emulator-5554"
    options.app = "/path/to/MyApp.apk"
    d = webdriver.Remote("http://127.0.0.1:4723", options=options)
    yield d
    d.quit()


def test_login_succeeds(driver):
    driver.find_element("accessibility id", "username").send_keys("alice")
    driver.find_element("accessibility id", "password").send_keys("hunter2")
    driver.find_element("accessibility id", "submit").click()
    assert driver.find_element("accessibility id", "welcome").is_displayed()

3. Run

pytest

Outputs land in:

test-results/
├── testrelic-timeline.json
└── testrelic-timeline.html

Configuration

Reporter options can be supplied three ways (highest priority first):

1. Environment variables (TESTRELIC_API_KEY, TESTRELIC_CLOUD_ENDPOINT, TESTRELIC_UPLOAD_STRATEGY, TESTRELIC_CLOUD_TIMEOUT, TESTRELIC_RUN_TYPE).
2. pytest.ini / pyproject.toml under testrelic_options.
3. .testrelic/testrelic-config.json in the project root.

pyproject.toml

[tool.pytest.ini_options]
testrelic_options = [
  "outputPath=./test-results/testrelic-timeline.json",
  "includeDeviceLogs=true",
  "includeNetworkLogs=true",
  "screenshotOnEvery=failure",
  "includeVideoRecording=false",
]

CLI flags

pytest --testrelic-output ./test-results/custom.json
pytest --testrelic-disable          # turn the reporter off for this run
pytest --testrelic-quiet            # suppress banner output
pytest --testrelic-no-autowrap      # disable WebDriver auto-detection

Full option list

Cloud integration

Set TESTRELIC_API_KEY and the reporter uploads the run to https://platform.testrelic.ai/api/v1/runs at session end.

export TESTRELIC_API_KEY=tk_live_...
export TESTRELIC_UPLOAD_STRATEGY=both   # realtime | batch | both | none

Or use a project file:

// .testrelic/testrelic-config.json
{
  "cloud": {
    "apiKey": "$TESTRELIC_API_KEY",
    "endpoint": "https://platform.testrelic.ai/api/v1",
    "upload": "both",
    "projectName": "my-mobile-app"
  }
}

Network failures are queued to .testrelic/queue/ and retried with:

testrelic-appium drain
testrelic-appium cleanup-queue --max-age-days 7

Entries the server rejects (a 4xx other than 401 / 403 / 408 / 429) move to .testrelic/queue/dead/ and are not retried. Transient failures (5xx, 408, 429, timeouts) stay queued and move to dead/ after 5 attempts. A drain stops early on a 401 / 403 or after 3 transient failures in a row, leaving the rest queued.

Parallel runs (pytest-xdist)

pytest -n 2 runs workers as separate subprocesses. The master process creates the cloud run during pytest_configure, writes a manifest, and finalizes the run after all workers exit. Each worker uploads its per-test realtime data under the master's run ID — exactly one run shows up on the dashboard.

CLI

testrelic-appium version
testrelic-appium merge shard-1.json shard-2.json -o merged.json
testrelic-appium serve ./test-results --port 9323
testrelic-appium drain
testrelic-appium cleanup-queue --max-age-days 7

Manual driver wrap

If auto-detection misses your driver (custom subclass, fixture indirection, non-pytest harness), use the explicit fixture or wrap function:

import pytest
from testrelic_appium import testrelic_appium_driver  # fixture

def test_with_explicit_fixture(testrelic_appium_driver):
    driver = testrelic_appium_driver  # already wrapped
    driver.find_element("accessibility id", "btn").click()

from testrelic_appium import wrap_driver

def test_with_manual_wrap(driver):
    unwrap = wrap_driver(driver)
    try:
        driver.find_element("accessibility id", "btn").click()
    finally:
        unwrap()

Compatibility

• Python 3.9 – 3.12
• pytest 7.0+
• Appium-Python-Client 4.x – 5.x
• Appium 2.x / 3.x

License

MIT — see LICENSE.