Skip to main content

testudos

A testing harness for Python packages using uv's isolated environments.

Overview

Testudos runs your test suite across every Python version that is both still supported upstream and compatible with your package. It works out which versions those are by intersecting two sources:

  1. Currently supported Python versions — from the endoflife.date API (v1)
  2. Your package's compatibility — the requires-python field in your pyproject.toml

Cycles that endoflife.date lists but that have no stable release yet (a Python still in alpha, beta or rc) are left out unless you ask for them with --include-prerelease.

In this document: how it works · choosing versions · execution · multiple packages · coverage · running in CI, with the GitHub Actions job and an organisation's baseline · exit codes · configuration reference

Quick start

# Install
uv tool install testudos

# See which versions would be tested, and why
testudos versions

# Run the suite on all of them
testudos run

# Run them concurrently
testudos run --parallel

# Show the exact commands without running anything
testudos run --dry-run

Installation

uv tool install testudos    # recommended
pip install testudos

Coverage needs nothing extra. testudos runs coverage through uv -- in each test environment, and on your machine to combine and report the data -- at one version for both, so the data files always read. It is pinned with testudos (currently 7.17.0); TESTUDOS_COVERAGE_VERSION picks another. Your uv exclude-newer cutoff does not apply to that one exact pin, so a cutoff older than its release still lets coverage install. On your machine it ignores the Python your project's .python-version names: a run's reports, which parse your code, use the newest Python the run tested, and everything else uses testudos' own. The coverage extra still installs, and does nothing.

Requirements

  • Python 3.11 or newer
  • uv 0.13.0 or newer: the first whose uv run --locked accepts the flags a coverage run passes, and the first that can download CPython 3.15.0. An older uv stops the run with exit 3 and says so, rather than failing every version's setup.
  • Linux or macOS. On other platforms testudos runs, but a timeout or an interrupt stops uv and can leave the test process it started running, and testudos warns about this when a run starts.

How it works

For each Python version, testudos runs your tests in an ephemeral environment created by uv run --isolated:

uv run --isolated --python=3.12 --with pytest --directory /path/to/project pytest
  • --isolated builds a throwaway environment, so nothing leaks between versions
  • --python=X.Y selects the interpreter, downloading it if needed
  • --with pytest puts the test runner inside that environment, where it can see your project's dependencies

--dry-run prints exactly these commands, built by the same code that runs them, so what you see is what would execute.

Test dependencies

uv run --isolated installs your project, its runtime dependencies, and its dev dependency group ([dependency-groups] dev), which uv includes by default. Other groups and extras are not installed unless you name them. If your test dependencies live in a group or an extra, name that: they are then installed at the versions in uv.lock.

testudos run --group test           # [dependency-groups] test
testudos run --extra test           # [project.optional-dependencies] test
[tool.testudos]
test-groups = ["test"]
test-extras = ["test"]

([tool.uv] default-groups = ["dev", "test"] in your project has the same effect for every uv run, testudos' included.)

For anything not declared in the project, name the package itself:

testudos run --with pytest-asyncio --with pytest-mock
[tool.testudos]
test-dependencies = ["pytest-asyncio", "pytest-cov>=7"]

These are installed outside the lockfile, so a requirement without == resolves to the newest release on every run: a new plugin release can change a run with no change to your project, and a freshly published release reaches every pipeline at once. A run warns about them before it starts, the JSON lists them as unpinned_dependencies, and --dry-run as "Unpinned --with". A package uv.lock pins is not counted: the environment already has it at its locked version. --require-pinned (require-pinned = true, TESTUDOS_REQUIRE_PINNED=1) refuses to run with any, exit 4. Prefer a group or an extra; otherwise pin them, or set uv's own exclude-newer (exclude-newer = "2026-10-01" in [tool.uv], or UV_EXCLUDE_NEWER) so resolution ignores anything released after that date. The same applies to a project with no uv.lock -- a Poetry or setuptools project, say -- whose whole dependency tree is resolved fresh each run, since uv does not read other tools' lock files.

The lockfile

With a uv.lock, each environment is installed from it. If pyproject.toml has changed since the lock was written, uv re-resolves in memory, says nothing and leaves uv.lock alone, so the run tests something other than what is committed. In CI you usually want that to fail:

testudos run --locked     # fail when uv.lock is out of date
testudos run --frozen     # use uv.lock as it is, without checking it

locked = true / frozen = true in [tool.testudos] do the same, and --no-locked / --no-frozen override them. UV_LOCKED=1 in the environment works too.

Other dependency versions

The lockfile is one answer to "which versions of my dependencies?". A library promises a range, and two other points of it are worth testing -- on a nightly or pre-release job rather than every pull request:

testudos run --resolution lowest-direct   # every direct requirement at its declared floor
testudos run --resolution highest         # the newest releases, ignoring uv.lock
testudos run --no-sources                 # published metadata, not [tool.uv.sources]

lowest-direct catches a floor that is too low -- requests>=2.20 while using an API added in 2.28 -- before a consumer with an older lock does. highest catches an upstream release that breaks you before your consumers resolve to it. --no-sources tests a workspace member against the releases its [project].dependencies allow rather than its siblings' working trees, which is what it will get once published; it pairs well with lowest-direct.

All three resolve anew, outside uv.lock (which is never written), so they cannot be combined with --locked / --frozen; a configured locked = true gives way to them. lowest-direct applies to --with requirements too, the test runner included: an unbounded pytest would be pytest's first release, so a run warns about any requirement nothing gives a floor -- give it one (test-dependencies = ["pytest>=8"]) or install it from a dependency group that has one. The dry run and its JSON say which resolution a run uses; resolution and no-sources in [tool.testudos], and TESTUDOS_RESOLUTION / TESTUDOS_NO_SOURCES, set them too.

The installed package

uv installs the project editable, so the suite imports it from the source tree, and a wheel that leaves a module or a data file out still passes. tox catches that by building the package first; --no-editable does the same:

testudos run --no-editable      # test the project installed as a built wheel

(editable = false in [tool.testudos], or TESTUDOS_EDITABLE=0.) Coverage works with it: the code under test ran from a temporary environment's site-packages, so it is measured by import name and the rcfile testudos writes maps */site-packages/ back to the source tree when the data is combined -- always, a single version included.

The wheel is rebuilt in each version's setup (uv would otherwise keep the one it built until pyproject.toml changes). A command that would put the project root first on sys.path -- python -m pytest, and any console script under --coverage, which runs it as coverage run -m pytest -- runs with PYTHONSAFEPATH=1, so it does not. Every process the suite starts inherits that: a test that runs python tools/script.py finds the script's own directory off the path too. Set PYTHONSAFEPATH= (empty) in the job to keep Python's usual path, at the cost of the source tree shadowing the wheel. A src/ layout makes this airtight; with the package at the project root, a conftest.py there can still put the root on sys.path and import the source tree after all, and so can a Python older than 3.11, which does not read PYTHONSAFEPATH.

Usage

Choosing versions

testudos run                                # auto-detect
testudos run --python 3.11 --python 3.12    # explicit (-p)
testudos run --python 3.12.3                # a specific patch release
testudos run --python 3.14.0rc1             # pre-releases are allowed explicitly
testudos run --python 3.14t                 # the free-threaded build (or 3.14+freethreaded)
testudos run --python pypy3.10              # PyPy
testudos run --include-prerelease           # include unreleased cycles when auto-detecting
testudos run --offline                      # skip the API, use cached or bundled data
testudos run --no-offline                   # refuse a configured offline = true
testudos run ./path/to/project              # a project other than the cwd
testudos --debug run                        # show the traceback behind a failure

testudos versions shows the same resolution as a table, marking each cycle stable or pre-release and naming any pre-release it left out. For scripts and CI it also prints JSON or a plain list:

testudos versions --format json     # {"versions": [...], "version_source": "api", "supported": [...]}
testudos versions --format plain    # 3.11 3.12 3.13 3.14
testudos run --discover --matrix package-version   # a GitHub Actions matrix, running nothing
uv python install $(testudos versions --format plain)

Free-threaded builds are where C extensions and thread-safety assumptions break first. include-variants = ["freethreaded"] adds the free-threaded build of every auto-detected cycle that has one (3.13 onwards) to the matrix; it can also be named directly, in --python, python-versions or TESTUDOS_PYTHON_VERSIONS. requires-python is matched against the language version, so 3.14t and pypy3.10 count as 3.14 and 3.10.

To hear early from the next Python, or a free-threaded build, without blocking merges on it, let it fail:

testudos run --include-prerelease --allow-failure prerelease
testudos run --allow-failure 3.15 --allow-failure freethreaded

allow-failure (config, or TESTUDOS_ALLOW_FAILURE) takes versions -- matched by cycle and build, so 3.15 covers 3.15.0b1 but not 3.15t -- and the words prerelease and freethreaded. Such a version runs and reports as usual: FAILED, ALLOWED in the table, "allowed_failure": true in the JSON, a ::warning rather than an ::error on GitHub. It does not change the exit code, even when --deadline keeps it from starting, and fail-fast does not stop on it.

Deciding the matrix centrally

endoflife.date moves on its own schedule, so the same commit can test a different matrix the day a Python ships or reaches end of life. An organisation that sets its own policy can name its supported cycles once, in a shared CI template or the runner environment:

export TESTUDOS_PYTHON_VERSIONS="3.11 3.12 3.13"   # stands in for endoflife.date

The list replaces endoflife.date (no network call is made) and is still intersected with each project's requires-python. A project's own python-versions, and --python, still win over it. version_source reports env. To keep endoflife.date's data but fetch it from an internal mirror of its API v1 response, set TESTUDOS_ENDOFLIFE_URL.

A floor below the matrix

Auto-detection tests only supported cycles, so a requires-python that admits end-of-life ones -- >=3.9 when 3.11 is the oldest supported -- tells installers the package works on Pythons nothing runs it on. testudos says so:

Warning: requires-python '>=3.9' admits 3.9, 3.10, past end of life, which
auto-detection does not test. Raise the floor, or name them in python-versions...

untested-floor in [tool.testudos] (or TESTUDOS_UNTESTED_FLOOR, an organisation's default that a project's own setting overrides) is warn, the default, ignore, or error, which stops the run with exit 4. testudos versions --format json lists them as untested_admitted. Named versions (--python, python-versions) are a deliberate choice and are not checked.

A member of a uv workspace is tested on the cycles every member's requires-python admits, since uv holds each member's interpreter to all of them: with a library at >=3.12 beside a service at >=3.13, the library is not tested on 3.12, and the same warning (or error) says so, naming the workspace's requires-python.

Refusing a guessed matrix

When neither endoflife.date nor the cache answers, testudos warns and tests its bundled list. On ephemeral CI runners there is never a cache, so a blocked or failed request takes that path. --strict-versions (config strict-versions = true, environment TESTUDOS_STRICT_VERSIONS=1) makes that an error instead.

Which interpreter

uv run --python=3.12 uses a 3.12 already installed on the machine if there is one, so the patch release tested depends on the runner image. uv's own python-preference decides that: UV_PYTHON_PREFERENCE=only-managed (or python-preference = "only-managed" in [tool.uv]) makes uv use its managed builds only, so every machine tests the same release. Newer uv releases also spell it UV_MANAGED_PYTHON=1, which is the one uv help run lists.

When requires-python rules out some releases of a cycle -- >=3.12.4 -- testudos asks uv for a release it allows (--python=>=3.12.4,==3.12.*) rather than for the cycle, so a machine whose own 3.12 is older (Ubuntu 24.04 ships 3.12.3) gets a managed 3.12 instead of a failed setup. A patch release you name (--python 3.12.3) is asked for as written.

Offline mode is also enabled by setting TESTUDOS_OFFLINE to anything except 0, false, no, off, n, f or the empty string (case-insensitive, and surrounding whitespace is ignored). Those values turn it off, overriding a configured offline = true; leaving the variable unset says nothing, so the configuration decides. --offline / --no-offline win over both.

Execution

testudos run --parallel                     # concurrent across versions (-P)
testudos run --parallel --max-concurrency 2 # at most two at once (-C)
testudos run --no-parallel                  # override a configured parallel
testudos run --no-fail-fast                 # keep going after a failure (-F)
testudos run --timeout 300                  # seconds per version (-t; default 1800)
testudos run --timeout 0                    # no timeout at all
testudos run --no-setup                     # skip the setup phase (below)
testudos run --deadline 50m                 # the whole run's time limit
testudos run --setup-timeout 1200           # seconds per version's setup (default 900)
testudos run --verbose                      # more detail (-v)
testudos run --quiet                        # suppress failed-test output (-q)
testudos run --dry-run                      # print commands, run nothing (-n)
testudos run -- -m "not slow" -x            # extra arguments for the test command

Everything after -- goes to the test command, after the configured test-args; --test-arg is the same one argument at a time, and TESTUDOS_TEST_ARGS sets them in a shared CI template (split the way a shell would, so -m "not slow" is two arguments). The command line wins over the variable.

Ctrl-C, SIGTERM and SIGHUP all stop a run the same way: every running test process is interrupted, given five seconds to finish its teardown, and then killed. testudos then reports what it has -- the results table, the --format json document (with "interrupted": true), the GitHub step summary and a JUnit file per version -- with every version that had not finished marked INTERRUPTED, and exits with code 2. Coverage is not combined. SIGTERM is what a cancelled CI job, docker stop or timeout(1) sends, so cancelling a pipeline does not leave suites running behind it, and does not throw away the hour of results that came before it. While those are written, a further SIGTERM or SIGHUP is held off; a second Ctrl-C still stops at once.

That holds when the signal reaches testudos. A CI system that runs a step through a shell may signal the shell alone -- GitHub's runner signals the bash that runs a run: script, and bash passes nothing on to the command it is waiting for -- so the job ends with no report and the suites still running. Start testudos with exec, as the last command of the step, so that it replaces the shell and is the process that is signalled; the examples below do.

Progress is a live table when stdout is a terminal. Anywhere else -- a CI log, a pipe, a file -- testudos prints one line as each version starts and finishes, and a "still running" line every minute while nothing finishes, so a long run never looks hung. FORCE_COLOR and --color add colour to those lines; they do not turn the live table on.

--deadline (or TESTUDOS_DEADLINE) bounds the whole run rather than one version: 90, 90s, 50m, 1h30m. When it passes, versions not yet started are recorded as skipped, running ones are stopped the way a timeout stops them, and the results are reported as usual -- set it a few minutes inside a CI job's own limit, which kills the job with nothing written. A run the deadline cut short exits 1, as a failing one does: not every version was tested in the time it had, and a retry would overrun the same way. The versions it cut short say so ("deadline_reached": true in JSON, beside "skipped": true for those that never started) and count as failed, not skipped, in the summary -- unless they are allowed to fail. On GitHub Actions the ones that never started are named in one error annotation. Reporting what ran gets two minutes past the deadline: coverage still reported for a run the deadline cut short, but no longer than that -- a package whose coverage is not reported by then fails it (a threshold nobody checked has not passed), and the JUnit reports, step summary and JSON are still written. So leave the job at least that much more than the deadline. An interrupt while results are being reported ends only the step it arrives in: the rest are written, and the run exits 2.

A version is over when its tests are. A process the suite started and never stopped -- a server a fixture left running -- is stopped once the tests have exited, rather than waited for, and the run warns naming the version ("stray_processes": true in JSON). The version's verdict is the suite's own exit code.

Every version runs under a timeout, 30 minutes unless you say otherwise (timeout = 0 in config, or --timeout 0, turns it off), so a hung suite stops well inside a CI job's limit rather than at it.

Before its tests, each version has a setup phase: the same uv run, with a one-line probe in place of the suite. It downloads the interpreter if needed and installs the environment into uv's cache, under its own timeout, so --timeout measures the tests and a slow first install never eats into it. The tests then run uv run --offline, from that cache: they reach no index, and get none of uv's index credentials (see Private indexes). Setup checks that they can, by running its probe once more offline; where the cache cannot rebuild the environment -- uv keeps none (UV_NO_CACHE), or a requirement cannot be resolved offline -- that version's tests run online, with a warning and "ran_online": true in JSON. The probe also reports the exact interpreter, shown in the results as 3.12 (3.12.11) and in JSON as "python" -- which patch release a cycle resolves to depends on the machine. When uv cannot provide the interpreter or install the environment, the version is reported as SETUP FAILED, with what uv said, and its tests are not run. --no-setup (config setup = false) skips the phase.

Fail-fast is on by default in sequential mode. Versions it skips are reported as SKIPPED rather than dropped, so the summary always accounts for every version you asked for. It does not apply in parallel mode, where every version is already running by the time the first failure is known.

Multiple packages

testudos run --package ./pkg1 --package ./pkg2   # explicit packages (-k)
testudos run ./monorepo --discover               # every package in the workspace (-d)
testudos run ./monorepo --discover -x 'examples/**'   # leave some out (--exclude)

--discover takes the workspace's own word for what its packages are: [tool.uv.workspace]'s members and exclude, or, for a monorepo uv does not manage, the same two globs under [tool.testudos.workspace]. Without either, every directory whose pyproject.toml declares a package ([project], [tool.poetry], or [build-system] beside a setup.py or setup.cfg, so not one holding a tool's settings alone) is a package, except what git ignores (build output, vendored trees). --exclude adds globs, relative to the root, for what is not a package -- test fixtures, examples, project templates. A workspace's own exclude is read as uv reads it, as globs from the root like members; --exclude is gitignore's, where a bare name matches at any depth. A project in the repository that the workspace's definition does not mention is not tested, and a warning names it. The root is tested when it declares a package itself; --exclude . leaves out one that is a project only to hold the workspace ([tool.uv] package = false), where a bare pytest would collect every member's tests without their environments.

A workspace root's [tool.testudos] is its members' defaults: a member inherits every setting it does not set itself, and a setting it does set replaces the root's outright, lists included. Only a root that declares a workspace is inherited from, so an unrelated pyproject.toml further up never reaches in. --dry-run names the inherited settings.

--package names the packages to test. The current directory is included only when you name it (testudos run . --package ./other), so the first example above tests exactly the two packages it lists.

Packages run alongside each other, and --parallel runs each package's versions alongside each other too. Each package keeps its own configuration, results and coverage data.

Those multiply, so a run is bounded by one number that does not:

testudos run ./monorepo --discover --parallel --max-concurrency 4   # (-C)

At most --max-concurrency uv processes run at any moment, however many packages and versions there are. It defaults to one per CPU; -C 1 tests one version of one package at a time. Without it, twenty packages across four Python versions would ask for eighty simultaneous environment builds, which on a laptop or a 2-core runner shows up as timeouts and OOM kills that look like flaky tests.

Suites that share something -- one database, a fixed port, a fixture directory -- must not overlap, across packages or across one package's versions alike: run them with -C 1.

A package's own max-concurrency setting applies when it is tested on its own. In a workspace the packages share one machine, so only --max-concurrency moves the limit.

The suites share the CPUs too. pytest-xdist's -n auto sizes its pool to every CPU, so four versions on four CPUs would run sixteen busy workers. testudos sets PYTEST_XDIST_AUTO_NUM_WORKERS in each test process to the usable CPUs divided by the number of suites running at once -- in a workspace, all of them, capped by --max-concurrency -- so -n auto (and -n logical) gets a fair share. An explicit -n 8 is untouched, and so is a value you set yourself.

Coverage

testudos run --coverage                                  # collect (-c)
testudos run --coverage --coverage-report html
testudos run --coverage --coverage-report html --coverage-report xml
testudos run --coverage --coverage-fail-under 80
testudos run --coverage --coverage-source mypackage      # default: your coverage config
testudos run --no-coverage                               # override a configured coverage

Formats: term, term-missing, html, xml, json, lcov. Data lands in .testudos/coverage/. .testudos/ holds what a run writes and ignores itself, with a .gitignore of its own, so it stays out of git status and of --changed-since.

What is measured is what your own coverage configuration chooses -- source, source_pkgs, source_dirs or [run] include -- read by coverage itself, from the same files in the same order as coverage run ($COVERAGE_RCFILE, .coveragerc, .coveragerc.toml, setup.cfg, tox.ini, pyproject.toml), so testudos measures the same files, and gets the same percentage, as your own coverage run or pytest --cov. Without such a setting, the packages your build backend declares are measured, else src/, else the importable packages at the project root. A suite whose own pytest options turn pytest-cov on (--cov in addopts) runs with --no-cov under --coverage: two coverages in one process leave testudos' with nothing measured. --coverage-source (or coverage-source in config) replaces your configuration's choice entirely, source_pkgs, source_dirs and include included; one of its entries that does not exist stops the run before any test starts (exit 4), since coverage would measure nothing there. An entry in your own configuration that does not exist is coverage's to warn about, as in your own runs. Whether branches are measured is your configuration's [run] branch as well: testudos does not turn it on.

The --coverage-fail-under threshold is checked against exact totals, so it gives the same verdict whichever report formats you asked for. Without one (the option, TESTUDOS_COVERAGE_FAIL_UNDER, or coverage-fail-under in [tool.testudos]), the threshold is your project's own: the --cov-fail-under in its pytest options -- the configuration pytest reads, which in a monorepo is often the repository root's, PYTEST_ADDOPTS, or the run's own test arguments -- which --no-cov would otherwise switch off with the rest of pytest-cov, else your coverage configuration's [report] fail_under. A project that gates on either stays gated. A threshold a coverage plugin sets when it loads (covdefaults' 100) is in neither place, so give testudos that one yourself: coverage-fail-under.

Tests that run in subprocesses -- pytest-xdist workers, multiprocessing -- are measured too: the run is given your project's coverage configuration plus coverage's patch = ["subprocess"], written to .testudos/coverage/coveragerc (edit your project's own config, not that file), which says what to measure as well. A run that measures 0% of its source says so on stderr.

Coverage is reported whenever data was collected, including when a version failed its tests. The two verdicts are independent and both feed the exit code, so a coverage regression cannot hide behind an unrelated test failure. A failing version still measured the code it ran; one that fail-fast never reached measured nothing, and the report names the versions it actually covers.

Coverage data can also be managed directly:

testudos coverage combine     # merge per-version data
testudos coverage report      # generate a report (--format; --fail-under, else the configured one)
testudos coverage clean       # remove .testudos/coverage

Combining keeps the per-version files, so it can be repeated and .coverage.3.11 is still there afterwards to answer "which lines does 3.11 miss that 3.13 covers?". testudos coverage clean is the only command that removes coverage data.

Machine-readable output

testudos run --format json
testudos run --format json --output results.json    # (-o)
testudos run --format json --dry-run

JSON output reports per-version results plus a summary of passed, failed and skipped counts, and version_source says whether the version matrix came from endoflife.date, the cache or the bundled fallback. With --coverage it carries a coverage block (totals, threshold, report paths); in multi-package mode it has one entry per package under packages, each shaped like the single-package payload. Errors and warnings go to stderr, so stdout is always one JSON document -- including on failure, when it carries "success": false and an error.

With --output, the document goes to the file and the terminal gets the same readable run as without --format json -- progress, the results table and the output of failed versions -- so a CI job can keep a JSON artifact and a log a person can read.

Private indexes

Indexes are uv's configuration, not testudos'. Every uv command a run makes -- each test environment, and the coverage testudos runs on the host to read your coverage configuration, combine and report -- inherits the environment and runs in the package's directory, so uv finds the same settings it would for a uv run of your own. With --coverage, nothing goes to PyPI behind your back.

Name the index where uv reads it. uv takes [tool.uv] from one file: in a uv workspace ([tool.uv.workspace]), the root's pyproject.toml, for every member; otherwise the first pyproject.toml with a [tool.uv] table, or uv.toml, going up from the package's directory. So in a monorepo uv does not manage ([tool.testudos.workspace]), an index in the root reaches only the members with no [tool.uv] of their own: one that has a table -- even just [tool.uv.sources] -- resolves from PyPI unless it names the index too.

[[tool.uv.index]]
name = "internal"
url = "https://pypi.internal.example.com/simple"
default = true            # in place of PyPI; without it, ahead of PyPI

Or set it for a job, which is how a shared CI template does it, and which reaches every uv testudos starts, whatever the layout: UV_DEFAULT_INDEX (in place of PyPI), UV_INDEX (more indexes, space-separated), UV_FIND_LINKS, UV_INDEX_STRATEGY. Both of the index variables take name=url (UV_DEFAULT_INDEX=internal=https://pypi.internal.example.com/simple), which names the index for its credentials. To install from find-links alone, set no-index = true in [tool.uv] or a uv.toml; uv has no variable for it. A [tool.testudos] key or TESTUDOS_* variable for one of these (default-index, TESTUDOS_INDEX, ...) is refused, exit 4, with a message naming uv's spelling.

Keep credentials out of the URLs. uv reads them from the environment for a named index (UV_INDEX_<NAME>_USERNAME, UV_INDEX_<NAME>_PASSWORD, the name upper-cased with anything else written _), from .netrc, or from a keyring, and the setup phase gets the environment as it is. A URL with credentials in it still works, and where testudos renders one -- a --with requirement or a test argument, in --dry-run, the JSON document or an error message -- its userinfo is written ****.

The suite never sees those variables. After setup, the tests run with uv run --offline from what setup put in uv's cache, so uv needs no credentials, and testudos leaves UV_INDEX_<NAME>_USERNAME, UV_INDEX_<NAME>_PASSWORD, UV_PUBLISH_*, UV_GITHUB_TOKEN and uv's API tokens (UV_API_KEY, UV_AUTH_TOKEN, PYX_*) out of the test process's environment, and takes the credentials out of the URLs in UV_DEFAULT_INDEX, UV_INDEX, UV_INDEX_URL, UV_EXTRA_INDEX_URL, the interpreter mirrors (UV_PYTHON_INSTALL_MIRROR, UV_PYPY_INSTALL_MIRROR) and UV_PUBLISH_URL. These are uv's own: another tool's variables (PIP_INDEX_URL, POETRY_HTTP_BASIC_*, TWINE_*) reach the suite as the job set them. A suite that needs a credential itself gets it under a name of its own (TEST_INDEX_TOKEN).

The tests run online, with the credentials, when uv's cache cannot rebuild the environment offline -- no cache to keep (UV_NO_CACHE, or no-cache in [tool.uv]), or a requirement uv cannot resolve offline -- and testudos warns naming the version ("ran_online": true in JSON). Under --no-setup they always run online with the credentials, and ran_online is not set: nothing fell back. Keep a uv cache, with setup on, if you want the credentials kept from the suite.

This keeps a token from being exposed by accident -- a test, a plugin or a crash report that dumps os.environ. It is not a defence against a hostile pull request: setup installs the project, which runs the pull request's build code (a setup.py, a hatch build hook, whatever [build-system] names) with the credentials in its environment. Nor does it cover a UV_FIND_LINKS URL's credentials (uv caches a find-links page under the URL as written, so they stay), an index URL written into pyproject.toml or uv.toml, or .netrc or a keyring, all of which the suite can read. A job that tests pull requests from outside the organisation should hold no credential that matters: a read-only, short-lived token, or none -- a token that can read a private index can often publish to it.

Colour

testudos --color run      # force colour
testudos --no-color run   # disable it

These are options on testudos itself, so they go before the subcommand. NO_COLOR and FORCE_COLOR are respected too.

Running in CI

GitHub Actions

testudos is a command, so a job is uv plus one step. Pin each action by commit SHA, and testudos and uv by version. The examples leave the pins to you: <sha> is the action's commit, <version> the testudos release (the newest on PyPI), <upload-time> a timestamp after that release's upload, such as 2026-11-08T00:00:00Z, and <uv-version> a uv release.

test:
  runs-on: ubuntu-latest
  timeout-minutes: 60
  permissions: { contents: read }       # the suite is code from the pull request
  env:
    TESTUDOS_DEADLINE: 55m              # stop and report inside the job's limit
    UV_PYTHON_PREFERENCE: only-managed  # the same interpreter on every runner
  steps:
    - uses: actions/checkout@<sha>
      with: { persist-credentials: false }   # keep the token out of .git/config
    - uses: astral-sh/setup-uv@<sha>
      with: { version: "<uv-version>", enable-cache: true }
    - run: >-                             # exec: a cancelled job's signal reaches testudos
        exec uvx --exclude-newer <upload-time> testudos@<version>
        run --parallel --no-fail-fast --strict-versions
        --junit-xml .testudos/junit --coverage --coverage-fail-under 80
    - uses: actions/upload-artifact@<sha>
      if: always()                      # failed and interrupted runs most of all
      with:
        name: testudos-junit
        path: .testudos/junit/
        include-hidden-files: true

The job owns everything around the run: services: (a database for integration tests), a container:, steps before it (system packages, a cloud login that yields a private index's token), a runner group.

To fan out, one job per package and Python version, so each is its own check and can be re-run on its own:

plan:
  runs-on: ubuntu-latest
  permissions: { contents: read }
  outputs: { matrix: "${{ steps.p.outputs.matrix }}" }
  steps:
    - uses: actions/checkout@<sha>
      with: { fetch-depth: 0, persist-credentials: false }   # history for --changed-since
    - uses: astral-sh/setup-uv@<sha>
      with: { version: "<uv-version>" }
    - id: p
      # Only what the pull request reaches. Empty on a push or a schedule,
      # which is no filter: the default branch is tested in full. Written
      # into the command, `--changed-since origin/main` would compare main
      # with itself, plan nothing, and pass without running a test.
      env:
        TESTUDOS_CHANGED_SINCE: "${{ github.base_ref && format('origin/{0}', github.base_ref) }}"
      # Assigned first: a command substituted into echo's argument loses its
      # exit code, so a failure would pass the step and skip the test job --
      # which required checks count as passing.
      run: |
        matrix=$(uvx --exclude-newer <upload-time> testudos@<version> run --matrix package-version --discover --strict-versions)
        echo "matrix=$matrix" >> "$GITHUB_OUTPUT"
test:
  needs: plan
  if: fromJSON(needs.plan.outputs.matrix).include[0] != null   # nothing to test
  runs-on: ubuntu-latest
  timeout-minutes: 30
  permissions: { contents: read }
  env:
    TESTUDOS_DEADLINE: 25m
    UV_PYTHON_PREFERENCE: only-managed
  strategy:
    fail-fast: false
    matrix: ${{ fromJSON(needs.plan.outputs.matrix) }}
  name: Test (${{ matrix.package }}, Python ${{ matrix.python }})
  continue-on-error: ${{ matrix['allow-failure'] }}
  steps:
    - uses: actions/checkout@<sha>
      with: { persist-credentials: false }
    - uses: astral-sh/setup-uv@<sha>
      with: { version: "<uv-version>", enable-cache: true }
    - env: { MATRIX_PATH: "${{ matrix.path }}", MATRIX_PYTHON: "${{ matrix.python }}" }
      # A --matrix package entry has `python: all`, which --python refuses: that
      # job resolves the versions itself, with the plan's --strict-versions.
      run: |
        set -- --python "$MATRIX_PYTHON"
        if [ "$MATRIX_PYTHON" = all ]; then set --; fi
        exec uvx --exclude-newer <upload-time> testudos@<version> run "$MATRIX_PATH" "$@" \
          --locked --strict-versions --junit-xml .testudos/junit
    - uses: actions/upload-artifact@<sha>
      if: always()
      with:
        name: junit-${{ matrix.id }}
        path: ${{ matrix.path }}/.testudos/junit/
        include-hidden-files: true
tests-pass:                          # the one check to require
  needs: [plan, test]
  if: always()
  runs-on: ubuntu-latest
  steps:
    - env: { PLAN: "${{ needs.plan.result }}", TEST: "${{ needs.test.result }}" }
      run: '[ "$PLAN" = success ] && { [ "$TEST" = success ] || [ "$TEST" = skipped ]; }'

The test jobs' names follow the matrix, so none of them can be a required check, and a test that was skipped -- nothing to test, or a plan that failed -- counts as passing. tests-pass is the stable name to require: it runs whatever happened, and passes when the plan succeeded and the tests passed or had nothing to run. It is a job of its own, so a minute of runner time on each run.

A test suite is code, and on a pull request it is the pull request's code: run it with permissions: { contents: read } and persist-credentials: false, so it can read neither a write token nor the one actions/checkout would leave in .git/config, and never under pull_request_target, which runs with the base repository's secrets. The matrix's paths come from the repository's tree -- the pull request's, on a pull request -- so the test step passes them to its script through env: rather than writing ${{ matrix.path }} into run:, where a shell would read them as code; workflow linters such as zizmor flag the inline form as template injection.

enable-cache keeps uv's cache between jobs, but not its interpreters, which live in uv's Python directory (UV_PYTHON_INSTALL_DIR): each job downloads the ones it tests. On GitHub-hosted runners leave it so: downloading them there is typically faster than restoring them from the Actions cache, which is why setup-uv leaves them out by default. Where downloads are slow or go through a mirror -- a self-hosted runner that starts empty, a firewall -- add cache-python: true, with a cache-suffix naming the uv version and, in the fan-out, the job's Python: "<uv-version>-${{ matrix.python }}". setup-uv saves an entry only when its key missed, and the key names neither (the Python in it is not the job's): without the Python, every job in the fan-out shares one entry, holding one job's interpreter; without the uv version, a new pin keeps restoring what the old one downloaded.

testudos run --matrix package-version (or package: one entry per package, with python: all and its versions in versions) prints {"include": [{"package", "path", "python", "id", "allow-failure"}, ...]} and runs nothing; every option a run takes applies, so --include-prerelease, --exclude, --package, --python and --allow-failure shape the matrix exactly as they would a run. An entry whose version may fail has "allow-failure": true, for continue-on-error. GitHub caps a matrix at 256 jobs; past that, package-version falls back to one entry per package and says so. A job given python: all runs without --python, as the test step above does, and resolves the versions itself, so pass it the options that shaped them in the plan: --strict-versions above, and --include-prerelease if the plan had it. --format json does not change this output: for the packages, their versions, which may fail and what --changed-since left out, run the same command with --dry-run --format json in place of --matrix.

A project whose coverage configuration names plugins is combined and reported from its own environment, where they are installed, so the job that combines needs what a test job needs: the project's index, and its lockfile.

With one job per version, each job's coverage covers one version, so code gated on the version reads as uncovered in every job. Run those jobs without a threshold -- --coverage --coverage-fail-under 0, since without the 0 a project's own threshold (coverage-fail-under, pytest-cov's, or coverage's fail_under) is applied to each job's one version and fails it. Keep each one's .testudos/coverage/ as an artifact named for the job (coverage-${{ matrix.id }}; the .coverage.* files, and the coveragerc that maps an installed package back to its source under --no-editable; upload-artifact needs include-hidden-files: true for them), and check the threshold once, in a job that downloads them into place. Into place is each package's own directory, <matrix.path>/.testudos/coverage/: every package's files have the same names, so downloaded into one directory they overwrite each other and one package's coverage is reported as the result. Then, for each package, testudos coverage combine <path> and testudos coverage report <path>, which applies the package's own threshold when --fail-under is not given. The report is rendered on the newest Python whose data it finds, as a run's are, since it parses your source and testudos' own Python may be older than its syntax: uv uses that Python if the job has it and downloads it otherwise.

Coverage data names each file by its absolute path, so the job that combines has to see the path the test jobs measured at. GitHub-hosted runners check every job out at the same one. Where they differ -- self-hosted runners with several work directories, GitLab's /builds/<token>/<n>/, Jenkins' workspace@2, Linux beside macOS -- one file from two jobs is reported as two, each partly covered, and the percentage is wrong. coverage's own [paths] setting maps them together, in the project's coverage configuration:

[tool.coverage.paths]
source = ["src/", "*/src/"]   # the first entry is where the files are now

For an organisation

A baseline for testing many repositories the same way, as environment variables a shared workflow sets:

concurrency: { group: "${{ github.workflow }}-${{ github.ref }}", cancel-in-progress: true }
permissions: { contents: read }
env:
  TESTUDOS_PYTHON_VERSIONS: ${{ vars.PYTHON_MATRIX }}   # one org-level variable decides the matrix
  TESTUDOS_STRICT_VERSIONS: "1"                          # never test the bundled guess
  TESTUDOS_LOCKED: "1"                                   # a stale uv.lock fails
  TESTUDOS_REQUIRE_PINNED: "1"                           # no test dependency outside the lock
  TESTUDOS_PARALLEL: "1"
  TESTUDOS_FAIL_FAST: "0"                                # one run reports every failing version
  TESTUDOS_JUNIT_XML: .testudos/junit                    # per-test annotations, and the reports
  TESTUDOS_DEADLINE: 55m                                 # inside timeout-minutes: 60
  UV_PYTHON_PREFERENCE: only-managed                     # the same interpreter on every runner
  UV_DEFAULT_INDEX: internal=${{ vars.INDEX_URL }}       # every uv command testudos runs uses it
  UV_INDEX_INTERNAL_USERNAME: ${{ secrets.INDEX_USER }}  # credentials via env, never in URLs:
  UV_INDEX_INTERNAL_PASSWORD: ${{ secrets.INDEX_TOKEN }} # setup uses them, the suite never sees them

This env: also reaches the uvx that installs testudos. Under --exclude-newer, uv skips every file whose upload time the index does not report (PEP 700), so from an index that reports none, uvx finds no testudos to install. A flag on uvx's own command line applies to uvx alone: uvx --default-index https://pypi.org/simple --exclude-newer <upload-time> testudos@<version>, or a mirror that reports upload times. Likewise, only-managed has uvx run testudos on a uv-managed Python, downloading one on a runner that has none.

What each line buys, and the rest of the baseline:

  • One matrix. PYTHON_MATRIX as an organisation variable, read by every repository; each project's requires-python still narrows it. Bump it once a year, when a cycle ships and one reaches end of life, and move the pinned uv with it: a uv released before the new cycle cannot download it.
  • Reproducible runs. Pin actions by commit SHA, and testudos and uv by version. Keep test dependencies in a locked group or extra (--group test) rather than --with; a project without a uv.lock resolves fresh every run, so give it uv's exclude-newer ([tool.uv], or UV_EXCLUDE_NEWER).
  • Time limits. timeout-minutes above the worst case, and a deadline a few minutes inside it, so a run that would overrun stops itself and reports.
  • Early warning, not blocked merges. --include-prerelease --allow-failure prerelease tests the next Python; include-variants = ["freethreaded"] with --allow-failure freethreaded the free-threaded builds.
  • The rest of the range, nightly. A scheduled job with --resolution highest catches upstream breakage before consumers do; one with --resolution lowest-direct --no-sources checks the floors a published library promises. Add --no-editable to a published library's jobs, so the wheel is what is tested.
  • Monorepos. --discover, --matrix package and --changed-since: one job per package, only for what a change reaches. Declare the workspace ([tool.uv.workspace], or [tool.testudos.workspace]), and put shared settings in the root's [tool.testudos], which members inherit.
  • Routing failures. Elsewhere than GitHub, use the exit code: 1 is the answer (tests, setup or coverage failed, or the deadline passed), 3 is worth a retry unless a version in the JSON has "uv_outdated": true (upgrade uv instead), 4 goes to whoever owns the configuration.

Whatever the CI system:

  • Pin testudos (uvx testudos@<version>): an unpinned install changes under you with each release. A release that changes or removes anything docs/STABILITY.md calls stable says so under ### Breaking in the CHANGELOG, with no deprecation period before it, so have automated bumps hold back a release with that section for a person to read. Pin its dependencies as well, by excluding anything published after the release: uvx --exclude-newer <upload-time> testudos@<version>, with any time after the release's upload to PyPI.
  • Pin uv (astral-sh/setup-uv with version:, or [tool.uv] required-version): uv is still 0.x, and its behaviour changes between releases. uv can only download the Pythons it knew of when it was released, so move the pin when the matrix grows -- each October, when endoflife.date lists a new cycle -- or the new version's setup fails, exit 3, in every repository at once. Or set TESTUDOS_PYTHON_VERSIONS (or python-versions) so the matrix grows only when you decide, and move both together.
  • --no-fail-fast, so one run reports every failing version.
  • --locked when uv.lock is committed, and test dependencies in a dependency group or extra (--group test) rather than --with, so they come from the lockfile; --require-pinned makes that a rule.
  • --strict-versions, or a central TESTUDOS_PYTHON_VERSIONS, so the matrix is never the bundled guess.
  • UV_PYTHON_PREFERENCE=only-managed for the same interpreter everywhere.
  • Access to uv's interpreter downloads: CPython from releases.astral.sh (Astral's mirror of python-build-standalone's GitHub releases), PyPy from downloads.python.org. Where a firewall blocks them, point UV_PYTHON_INSTALL_MIRROR (CPython) and UV_PYPY_INSTALL_MIRROR (PyPy) at a mirror of those downloads, or a setup that has to download an interpreter fails.
  • The default 30-minute timeout, or a --timeout sized for your suite; the setup phase keeps interpreter downloads out of it.
  • --format json --output results.json for a machine-readable record next to the readable log, and the exit code (below) to tell a failing suite from an outage. Every JSON document says what produced it -- "generated_by": {"testudos": ..., "uv": ...} -- and so does the step summary; testudos --version prints both for a log.

On GitHub Actions (GITHUB_ACTIONS=true, which the runner sets) testudos also writes what GitHub reads from a job:

  • each version's output inside a ::group::, so the log folds to one line per version, and inside a ::stop-commands:: block with a random token, so a line the suite prints that starts with :: -- a fake ::error, an ::add-mask:: -- is shown as text rather than obeyed;
  • an ::error annotation for each failing test, on its line in the pull request's diff and naming every version it fails on (tests.test_x::test_parse -- fails on 3.12, 3.13): a test that fails everywhere is one annotation, not one per version. These, and the grid in the step summary below, are read from the JUnit reports, so they need --junit-xml (or TESTUDOS_JUNIT_XML, or junit-xml in [tool.testudos]), as in the examples above; without it, a failing version gets one annotation for the whole version;
  • an ::error annotation for each failed version the tests do not account for (a setup failure, a uv too old to download that Python, a timeout, a stop at --deadline, a suite that is not pytest), one naming every version --deadline kept from starting, one for a missed coverage threshold, and a ::warning for each warning (a fallback matrix, an untested floor). These show on the run's summary page and the pull request's checks. GitHub shows at most ten error annotations per step, so the tests failing on the most versions come first and the last annotation says how many more there are;
  • a results table appended to $GITHUB_STEP_SUMMARY, with a grid of the failing tests by Python version (❌ failed, ✅ passed, – not run), cut to fit GitHub's 1 MiB per step -- the tests failing most widely kept, and a line saying how many more there are.

--ci-format github turns this on anywhere, --ci-format none turns it off.

testudos run --junit-xml .testudos/junit     # a JUnit XML report per version

For pytest, each version writes .testudos/junit/<version>.xml itself, with a suite named python-<version> -- <package>/python-<version> when the run has more than one package, so a report viewer reading every file keeps them apart -- which test-report actions and GitLab or Jenkins read. A version pytest did not write one for -- its setup failed, it timed out, fail-fast skipped it, --deadline kept it from starting (a failure, unless it is allowed to fail), or the suite is not pytest -- gets a one-test-case report from testudos, so no version is missing. A relative directory is inside each package; an absolute one gets a subdirectory per package, named after the package (two packages can share a directory name, never a package name). The paths are in the JSON as "junit_xml".

In a monorepo, test only what a change reaches:

testudos run --discover --changed-since origin/main

A package is tested when a file inside it changed since the branch left origin/main (committed, uncommitted or untracked), or when it depends on one that did -- through [project].dependencies, optional dependencies, dependency groups, [tool.uv.sources], Poetry's dependency tables and groups, or the requirement files setuptools reads for dynamic dependencies (-r and -e <path> included), or a local path in testudos' own test-dependencies or --with -- directly or through other packages, those the run leaves out (--exclude, a --package subset) included. A package whose dependencies are dynamic with nothing readable behind them (a setup.py computing them) is assumed to depend on every package. A change outside every package (CI configuration, say) tests all of them, and so does a change a package owns that none of the packages nested inside it does: when the repository root is itself a package, its uv.lock or conftest.py reaches every member. The rest are listed as not affected, and as "not_affected" in JSON, with the reason each tested package was picked in "affected_by" (and in --verbose output); with nothing affected the run exits 0.

Rules in the repository root's pyproject.toml refine that, as globs written the way .gitignore writes them:

[tool.testudos.changed-since]
ignore = ["*.md", "docs/**"]        # these changes affect no package
global = ["libs/shared/fixtures/**"] # these affect every package

It needs the ref's history: actions/checkout with fetch-depth: 0. Compare a pull request against its base (origin/${{ github.base_ref }}) and a merge queue entry against github.event.merge_group.base_sha; leave changed-since empty on pushes to the default branch and on scheduled runs, so the default branch is always tested in full. A ref that cannot be compared against -- the null commit GitHub reports as github.event.before for a push that creates a branch, a commit a force-push left behind, a ref a shallow clone lacks -- tests every package, with a warning saying why, rather than failing the run.

Exit codes

Code Meaning
0 Every version passed, and coverage met its threshold
1 Tests failed, an environment could not be set up (a stale lock under --locked, requirements that do not resolve, a build that fails), --deadline passed before every version had run, or coverage missed --coverage-fail-under
2 Interrupted: Ctrl-C, SIGTERM or SIGHUP
3 testudos could not run: uv missing or older than 0.13.0, a uv with no download for a version's Python (one released after that uv) when nothing else failed and no configuration is wrong, no version data under --strict-versions, an unwritable --output, or testudos itself crashed
4 The command line or a configuration is wrong: an unknown option or value, an invalid [tool.testudos], a requires-python nothing satisfies, a missing coverage source

CI can retry on 3 -- unless a version in the --format json results has "uv_outdated": true, which a retry on the same uv repeats -- send 4 to whoever owns the configuration, and treat 1 as the result. A setup failure is a 1 rather than a 3 because retrying it almost never helps: the lockfile, the requirements or the build is what needs fixing. Nor does retrying a run that ran out of time: it overruns the same way, so a missed deadline is a 1 too, whether it stopped a version or kept one from starting. The one setup failure that is a 3 is a Python the uv in use cannot download: uv embeds its list of Python downloads, so a cycle released after that uv is out of its reach, and the uv is what needs changing -- as with a uv older than 0.13.0, a retry on the same uv fails the same way. testudos marks the version ("uv_outdated": true in JSON), names the uv version and says to upgrade it, when uv says it has no download for that Python and its own list of Pythons agrees: the cycle is later than every one it can download, and none is installed. A Python older than uv's downloads, or a patch release that may not exist (3.12.99), stays an ordinary setup failure. Such a version does not stop --fail-fast, and coverage expects no data from it. Beside a real failure or a missed deadline the run is still a 1, and beside a configuration error a 4.

Environment variables

Most run options can be set from the environment, which is how shared CI templates are usually configured; a flag on the command line still wins. testudos run --help names each option's variable (TESTUDOS_PARALLEL, TESTUDOS_TIMEOUT, TESTUDOS_COVERAGE_FAIL_UNDER, ...), and is the list to trust. Repeatable options take a space-separated list (TESTUDOS_GROUP="test lint"). TESTUDOS_UNTESTED_FLOOR (warn, ignore, error) is read below a project's own untested-floor. TESTUDOS_OFFLINE, TESTUDOS_STRICT_VERSIONS, TESTUDOS_PYTHON_VERSIONS, TESTUDOS_ENDOFLIFE_URL, TESTUDOS_COVERAGE_VERSION and TESTUDOS_TRACEBACK are described with their features above. uv's own variables (UV_DEFAULT_INDEX, UV_EXCLUDE_NEWER, UV_PYTHON_PREFERENCE, ...) reach every uv command testudos starts.

Proxies and certificates

testudos makes one HTTP request, to endoflife.date, with Python's standard library: it honours HTTPS_PROXY and NO_PROXY and verifies TLS against the operating system's trust store, so a TLS-inspecting corporate proxy whose certificate the machine trusts works without extra configuration. uv has its own settings for the package index (UV_NATIVE_TLS / UV_SYSTEM_CERTS).

Configuration

Everything above can be set in [tool.testudos]. A key it does not have is an error (exit 4) naming the nearest one that it does, so a mistyped strict_versions cannot leave the check off:

[project]
requires-python = ">=3.11"

[tool.testudos]
# Version selection
python-versions = ["3.11", "3.12", "3.13"]   # override auto-detection
include-prerelease = false                    # include unreleased cycles
include-variants = ["freethreaded"]           # add 3.13t, 3.14t, ... to auto-detection
allow-failure = ["prerelease", "freethreaded"] # report these, but do not fail on them
offline = false                               # never call endoflife.date
strict-versions = false                       # fail rather than test the bundled list

# What to run
test-command = "pytest"
test-args = ["-v", "--tb=short"]
test-groups = ["test"]                        # [dependency-groups] to install
test-extras = ["test"]                        # extras to install
test-dependencies = ["pytest-asyncio"]        # extra packages per isolated env
timeout = 300                                 # seconds per version (0: none)
setup = true                                  # install each environment first
setup-timeout = 900                           # seconds per version's setup

# The lockfile
locked = true                                 # fail when uv.lock is stale
frozen = false                                # use uv.lock without checking it
require-pinned = false                        # refuse unpinned --with requirements
untested-floor = "warn"                       # requires-python admitting EOL versions: warn, ignore, error
resolution = "locked"                         # or highest / lowest-direct, resolved outside uv.lock
no-sources = false                            # ignore [tool.uv.sources]
editable = true                               # false: test the built wheel

# How to run it
junit-xml = ".testudos/junit"                 # a JUnit XML report per version
parallel = false
max-concurrency = 8                           # uv processes at once (default: CPUs)

# Coverage
coverage = false
coverage-report = ["term"]
coverage-fail-under = 80
coverage-source = ["src"]

# --changed-since rules (read from the repository root's pyproject.toml)
# The workspace's packages, for a monorepo uv does not manage
[tool.testudos.workspace]
members = ["libs/*", "apps/*"]
exclude = ["libs/legacy"]

[tool.testudos.changed-since]
ignore = ["*.md", "docs/**"]                  # changes that affect no package
global = []                                   # changes that affect every package

CLI options win over configuration. Values are validated on load: a wrong type or an out-of-range value is an error, and so is an unrecognised key. Indexes, exclude-newer and python-preference are uv's settings, in [tool.uv] or uv.toml (see Private indexes); the [tool.testudos] keys of those names are an error.

test-command is a command line, not just a program name, so a runner that needs arguments to name itself fits in one setting:

test-command = "pytest"
test-command = "python -m unittest discover"
test-command = "./run-tests"

It is split the way a shell would split it (quoted words stay together), but no shell runs it: the pieces become an argv list. Only the first word is a package name, so it is what gets installed into each isolated environment — a suite run as python -m pytest should name pytest in test-dependencies, since the interpreter is what testudos sees. A first word containing a /, like ./run-tests, is a file in your project rather than a package, so nothing is installed for it.

test-args are passed to that command exactly as written, with no filtering, so metacharacters need no escaping and no workaround:

test-args = ["--ignore-glob=*$py.class", "-W", "error:.*deprecated$:UserWarning"]

By the same token nothing in them is expanded either — --rootdir=$PWD arrives as the literal string $PWD, not as your working directory.

Documentation

  • Architecture — how the pieces fit together, and the decisions behind them
  • Stability — what a version bump can and cannot change, and how breaking changes are announced

Development

git clone https://github.com/martinristovski/testudos.git
cd testudos
uv sync --extra dev

just ci            # lint, type-check, test, network tests, dogfood, changelog check
just test          # pytest with coverage
just fix           # auto-fix lint and formatting
just dogfood       # run testudos on itself

The default test suite is hermetic: the endoflife.date API is stubbed and the version cache is redirected per test, so no test touches the network or your real ~/.cache/testudos.

A separate set of tests does reach the network: most exercise the real uv boundary, and so resolve packages from PyPI, and one checks the canned endoflife.date response against the live API. They are marked network and deselected by default; run them with pytest -m network. CI runs them as their own step, so an index or API outage fails a step that names the cause.

License

MIT

Metadata

Release files for testudos 0.74.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for testudos 0.74.0
File Size Uploaded Provenance Actions
testudos-0.74.0.tar.gz 553.9 kB With provenance Details for testudos-0.74.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for testudos 0.74.0
File Interpreter ABI Platform Actions
testudos-0.74.0-py3-none-any.whl Python 3 none any Details for testudos-0.74.0-py3-none-any.whl

Total release size: 754.8 kB

Release files / testudos-0.74.0.tar.gz

Download URL testudos-0.74.0.tar.gz
Size 553.9 kB
Tags Source
SHA-256 checksum
How to use checksums
251ceaf9ab10e54a0a15f3dcccffc905a83c297bd526d115d267d306aa78e92c
BLAKE2b-256 checksum
How to use checksums
329a68fb55f8c4a68537117c3b59eca96e9a09c4533e22b6eaca7d039f9c8caa
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.

Transparency log

Release files / testudos-0.74.0-py3-none-any.whl

Download URL testudos-0.74.0-py3-none-any.whl
Size 200.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
49795c0c4d4670f3b1257accea2ab9021b8f4055dda032f3b291ee8e4e85c549
BLAKE2b-256 checksum
How to use checksums
c4c3c7edd1915d22cdea2d386f0b88ccd8419963fc6c2ff849981102a90c5217
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.

Transparency log

Security for testudos 0.74.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for testudos-0.74.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.74.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page