Testosterone

See what broke, what's flaky and what got slower on every run. Test history, run-to-run diffs and flaky detection for pytest, Playwright, Jest, go test or any runner that writes JUnit XML: on your laptop, in CI, or on one shared server for the whole team.

Live demo: https://taltal-beep.github.io/testosterone/ · Docs: https://taltal-beep.github.io/testosterone/docs/

Run detail (failed fake-api run)	Compare (two fake-api runs)
Cycle page with its flaky tests	Dashboard on a phone

What you're looking at: Testosterone testing itself (its own test suite) and fake-api, a small app whose routes fail on purpose. CI runs both cycles and publishes the real results as a read-only static site.

What's new

New in 1.2.0:

• testo ingest: bring any runner. Keep your CI as it is and add one step after the tests: testo ingest 'reports/**/*.xml' --cycle web. JUnit XML from Jest, Vitest, go test, JUnit5 or anything else becomes a recorded run with history, Compare, testo flaky and the PR comment. No testosterone.yaml needed.
• Team server. docker compose up in deploy/server/ starts the API, the web UI and Postgres in one place; CI jobs upload with testo ingest --server URL --token ... so the whole team shares one history. Guide: Team Server.
• Lighter install. pip install testo-core no longer pulls in Behave, BehaveX and Allure-Behave. Behave users add the extra: pip install 'testo-core[bdd]'.
• Flaky test detection. testo flaky --cycle <name> lists tests that keep flipping between pass and fail across a cycle's recent runs (--fail-on-flaky to gate CI), and the cycle page in the UI shows them too.
• Pull request comments. The GitHub Action writes the run, its failing tests and what changed since the previous run to the job summary, and with comment-on-pr: "true" keeps one comment per cycle on the pull request. The text comes from testo report markdown.
• Plugins. Add your own test runner or reporter from any package through the testo.frameworks / testo.reporters entry points; testo plugins lists what is installed. examples/plugin/ is a complete one.
• Playwright. equipment: playwright runs npx playwright test with its JUnit and HTML reporters wired in, so its tests count in summaries, diffs and the UI with no extra setup.
• A restyled web UI (navy glass cards, a sidebar on desktop) and a docs site with a Getting Started guide.
• Fixes: a pip installed testo-core keeps its run data in ./.testo/ (or $TESTO_HOME) instead of site-packages, and BehaveX stages keep BehaveX's own full HTML report.

Full list in CHANGELOG.md.

Why

After a red build the question is never just "what failed?". It's what changed since the last green run, is this test flaky, and did anything get slower? CI logs and one-off HTML reports can't answer that, because they forget every run as soon as it ends. Testosterone records every run and answers it: per-test regressions and fixes between runs, flaky tests, durations over time, a test pyramid and an optional AI summary of the failures.

There are two ways in. testo ingest records JUnit XML that your existing CI already writes, so adoption is one extra step. testo run goes further: it runs several frameworks as one cycle of stages from a testosterone.yaml (pytest, Playwright, Behave, any command), the same way from the CLI, CI or the web UI. Point either at a shared team server and every CI job lands in one history.

How it compares. Allure renders one run; ReportPortal is a large server to operate. Testosterone is a pip install that keeps the history and diffs runs against each other, works with whatever runner you have, and can still hand results to Allure, ReportPortal, Extent or TestBeats as reporters.

What ships: the testo CLI (--ci streams NDJSON events, exit codes 0–4), a FastAPI backend (testo_api, /api/v1, live runs over server-sent events), a React frontend (frontend/), and GitHub Action / GitLab CI wrappers around testo run --ci. See ARCHITECTURE.md for how the pieces fit.

Quickstart

Install the CLI from PyPI and point it at your own tests:

Already have tests that write JUnit XML? Record them, no config needed:

pip install testo-core
npx jest --ci --reporters=default --reporters=jest-junit   # or pytest --junitxml, go test, ...
testo ingest 'junit*.xml' --cycle web                      # recorded; run again to see the diff
testo summary --cycle web

Or let Testosterone run the tests as a cycle:

pip install testo-core            # the `testo` CLI; add [api] for the REST API, [db] for Postgres,
                                  # [bdd] for Behave and BehaveX
testo config init                 # writes a starter testosterone.yaml; edit its stages
testo run --cycle smoke-test      # run a cycle from it

An installed testo keeps per-run report copies in ./.testo/ (set TESTO_HOME to move them).

Getting Started goes further in about five minutes: a regression between two runs, then the same cycle in GitHub Actions.

Or work from a checkout of this repository, which ships sample cycles:

git clone https://github.com/taltal-beep/testosterone.git
cd testosterone
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

testo run --cycle sample-pytests      # run a cycle from testosterone.yaml
testo report --cycle sample-pytests   # build the Allure report for it

Adding Testosterone to your own project with an AI coding assistant? Paste one of the Setup Prompts.

Runs are stored in SQLite by default (database.url in testosterone.yaml, or DATABASE_URL). Postgres and MySQL work too; docker compose up -d postgres starts one.

UI

testo-api                              # FastAPI on http://127.0.0.1:8000
npm --prefix frontend install
npm --prefix frontend run dev          # React on http://localhost:5173

Pages: Dashboard, Cycles (run any cycle and watch its stages live, see its flaky tests), Runs and Run Detail (reports, test pyramid, AI summary), Compare (delta between two runs), Quick Run (run one framework against a repo without defining a cycle) and AI settings.

Defining cycles

version: 1
defaults:
  target_repo: .
  artifacts_root: artifacts
  timeout_s: 600
reporters:
  - type: allure
cycles:
  smoke:
    description: Unit tests, then BDD flows.
    trigger:                     # optional: skip unless these paths changed
      paths: ["src/**", "tests/**"]
    stages:
      - name: unit
        equipment: pytest
        args: [-q, tests/unit]
      - name: flows
        equipment: behave
        args: [features/smoke.feature]   # always target Behave features explicitly
      - name: browser
        equipment: playwright            # npx playwright test, JUnit + HTML report wired in
        args: [tests/e2e]
      - name: web
        equipment: command               # any runner; results come from JUnit XML
        args: [npx, jest, --ci]
        junit_xml: ["reports/junit/*.xml"]

Built-in equipment: pytest, behave, behavex, playwright, command and junit (records JUnit XML another step wrote; what testo ingest uses). behave and behavex need pip install 'testo-core[bdd]'. A package can add more (see Plugins).

testo config validate checks the file; testo init writes one interactively. The full schema and every flag are in Command Reference.

CLI

Exit codes: 0 success (or trigger-skipped), 1 tests failed, 2 invalid input, 3 infrastructure failure, 4 internal error. Details: Troubleshooting and Error Codes.

API

Interactive docs at http://127.0.0.1:8000/docs while testo-api runs.

CI

GitHub Actions:

- uses: taltal-beep/testosterone/integrations/github-action@v1
  with:
    cycle: smoke          # optional: config-path, ci-mode, persist, python-version
    comment-on-pr: "true" # results as a pull request comment (needs pull-requests: write)

GitLab CI (a GitLab mirror of this repo is coming; until then, include the template straight from GitHub, or copy ci/gitlab/testo.gitlab-ci.yml):

include:
  - remote: "https://raw.githubusercontent.com/taltal-beep/testosterone/v1/ci/gitlab/testo.gitlab-ci.yml"
variables:
  TESTO_CYCLE: "smoke"

Both run testo run --ci, keep the NDJSON event stream, and expose the final plan_finished event (exit code, per-stage results). The GitHub Action also writes the run to the job summary (job-summary, on by default) and, with comment-on-pr, keeps one up-to-date comment per cycle on the pull request: results per stage, failing tests and per-test changes since the last run. Run records written in CI carry the provider, pipeline, job, commit and branch. Inputs and outputs: integrations/github-action/README.md.

Optional integrations

• Metrics push: after each run, test KPIs go to InfluxDB when INFLUXDB_URL, INFLUXDB_TOKEN, INFLUXDB_ORG and INFLUXDB_BUCKET are set, and to a Prometheus Pushgateway when PROMETHEUS_PUSHGATEWAY_URL is set (PROMETHEUS_JOB_NAME defaults to testo). Pushes are best-effort and never change the run result.
• Reporters: allure, extent, reportportal, testbeats under reporters:.
• Plugins: a package adds a test runner or a reporter through the testo.frameworks / testo.reporters entry point groups, with no change to Testosterone; testo plugins lists and checks them. examples/plugin/ adds a unittest runner and a json-summary reporter.
• AI failure summaries: opt in on the AI settings page with an OpenAI or Anthropic key, or point it at any OpenAI-compatible endpoint (a free tier such as Groq or Gemini, or a local Ollama). Failed runs store their failing cases, first traceback and log tail (redacted), which is what the summary is built from.

Development

pip install -e ".[dev]"
pre-commit install
ruff check .
ruff format --check .
mypy testo_core testo_api
pytest -q -m "tier_fast and not quarantined" --no-cov
npm --prefix frontend run typecheck
npm --prefix frontend test

ruff check, ruff format --check and mypy testo_core testo_api are all blocking in CI (.github/workflows/ci.yml's format job), as are the frontend typecheck and tests.

See CONTRIBUTING.md for the full checklist. Start with ARCHITECTURE.md for the design; docs/ holds the design notes and ADRs, also published as the docs site (pip install -e ".[docs]" and mkdocs serve to preview it).

Contributors

Thank you to everyone who has helped make Testosterone better:

Want to join them? The good first issues are a friendly place to start, and CONTRIBUTING.md walks you through the setup.

History

The project started as UQO ("Unified Quality Orchestrator") and was renamed to Testosterone (the uqo command and UQO_* variables became testo and TESTO_*). v1.1 replaced the Streamlit UI with the React frontend and dropped the Docker-based headless runner (stages now run as host subprocesses). Full details in CHANGELOG.md.