test-reporting 3.0.0

Multi-project test reporting dashboard — collect results locally or push to S3

test-reporting

Multi-project test reporting dashboard. Collects pytest results, publishes to a central store (local folder or S3), and serves a clean dashboard anyone can read.

---

How it works

Project A tests run  ──┐
Project B tests run  ──┼──► Central store (local or S3) ──► Dashboard (index.html)
Project C tests run  ──┘

Each project publishes its results to a shared location. The dashboard reads from that location and shows all projects together. The HTML files are static — they're deployed once and the data file (data.json) is updated after every run.

---

Installation

pip install test-reporting

# With S3 support
pip install test-reporting[s3]

---

Quick start

1. Add reporting.ini to your project

[reporting]
project_name = my-project

store_type = local
store_path = /shared/test-reports

# auto_publish = true   # push results automatically after every test run

2. Deploy the dashboard (once, from any project)

test-report deploy

This copies the static HTML files to store_path. Only needs to be done once.

3. Run tests and publish

pytest
test-report publish

Or set auto_publish = true in reporting.ini to skip the manual publish step.

4. Open the dashboard

# Local
open /shared/test-reports/index.html

# S3 — your bucket's static website URL

---

Multi-project setup

Every project has its own reporting.ini with a unique project_name, all pointing to the same store_path or S3 bucket.

Project A:  project_name = frontend-tests,  store_path = /shared/reports
Project B:  project_name = api-tests,       store_path = /shared/reports
Project C:  project_name = e2e-tests,       store_path = /shared/reports

The dashboard aggregates all three automatically.

---

Test suite structure

The plugin detects the suite name from the pytest argument automatically:

pytest tests/abc/test_suite_abc.py   # suite_name = test_suite_abc
pytest tests/abc/test_file_a.py      # suite_name = test_file_a

Suites run on the same day are grouped as a regression in the dashboard.

Tagging regressions explicitly

Set REGRESSION_TAG as an environment variable when kicking off your suites. All runs with the same tag are grouped together:

# Jenkins — set in each job's environment
REGRESSION_TAG=sprint-42 pytest tests/abc/test_suite_abc.py

# Local
export REGRESSION_TAG=sprint-42
pytest tests/abc/test_suite_abc.py
pytest tests/def/test_suite_def.py

Or set it in reporting.ini:

regression_tag = sprint-42

---

Jenkins setup

Local store (shared network drive)

[reporting]
project_name = my-project
store_type = local
store_path = //network-share/test-reports

stage('Test') {
    sh 'pytest tests/'
}
stage('Publish') {
    sh 'test-report publish'
}

S3

[reporting]
project_name = my-project
store_type = s3
s3_bucket = my-test-reports-bucket
s3_prefix = reports/
s3_region = us-east-1

AWS credentials are read from environment variables or an IAM role — not stored in reporting.ini.

environment {
    REGRESSION_TAG = "2026-05-29"
    AWS_ACCESS_KEY_ID     = credentials('aws-access-key')
    AWS_SECRET_ACCESS_KEY = credentials('aws-secret-key')
}

stage('Test') {
    sh 'pytest tests/'
}
stage('Publish') {
    sh 'test-report publish'
}

---

Configuration reference

[reporting]
# Required
project_name = my-project

# Store
store_type = local          # local | s3
store_path = /shared/reports  # for store_type = local
# s3_bucket = my-bucket     # for store_type = s3
# s3_prefix = reports/
# s3_region = us-east-1

# Auto-publish after every test run (default: false)
auto_publish = false

# Regression grouping tag (can also be set via REGRESSION_TAG env var)
# regression_tag = sprint-42

# Local SQLite — used by stats/suites/cleanup commands
db_path = test_results.db
retention_days = 365

---

CLI commands

Command	Description
test-report publish	Push latest test run to the store
test-report deploy	Deploy static dashboard HTML to the store (run once)
test-report open	Open local dashboard in browser
test-report stats	Show stats from latest run (reads local SQLite)
test-report suites	Show suite statistics (reads local SQLite)
test-report cleanup	Delete old runs from local SQLite

---

Dashboard pages

index.html — All projects at a glance. Pass rate, health status, last run time.

project.html?p=<name> — Single project view. Regression history grouped by tag or date, suite run history, trend chart.

run.html?r=<id>&p=<name> — Run detail. Test file breakdown, every failing test with its error message, screenshot and trace links.

---

Migrating from v1

1. pip install --upgrade test-reporting
2. Add store_type, store_path (or S3 config) to reporting.ini
3. Run test-report deploy once
4. Replace test-report generate with test-report publish in your pipeline

Old commands that still work: stats, suites, cleanup

---

Project structure

reporting/
  plugin.py       — pytest plugin (auto-collects results)
  publisher.py    — writes run files + rebuilds data.json
  config.py       — reads reporting.ini / env vars
  classifier.py   — categorises failures
  storage.py      — local SQLite
  cli.py          — CLI entry point
  templates/
    index.html    — projects landing page
    project.html  — per-project view
    run.html      — run detail view

---

Requirements

• Python >= 3.9
• pytest >= 8.0.0
• boto3 >= 1.26.0 (only for S3 — pip install test-reporting[s3])

---

License

MIT