testcost

Find out where your test suite's time actually goes.

pytest --durations ranks tests by how long each one took. That answers the wrong question most of the time: a four minute suite rarely contains a four minute test. It contains a session fixture that builds a database, or three hundred modules imported during collection, or a function-scoped fixture that costs 20ms and runs two thousand times.

testcost attributes the time to collection, imports, fixture setup and teardown, and the test bodies themselves, so the largest line in the report is the thing worth fixing.

Installation

pip install testcost

Requires Python 3.10+ and pytest 8+.

Quick start

testcost run -- tests

4m12.3s for 1,284 tests
18.4s of that is importing, pytest included

seconds  share  where
-------  -----  ------------
 2m41.0s   64%  fixtures
   58.2s   23%  tests
   19.8s    8%  collection
   13.3s    5%  unattributed

fixtures
 total  runs   each  scope     fixture
------  ----  -----  --------  --------------------------------
1m52.0s     1  112s  session   postgres_container  tests.conftest
  38.4s  1920   20ms  function  db_transaction      tests.conftest
  10.6s    64  166ms  module    seeded_catalogue    tests.fixtures.data

worth a wider scope
  db_transaction costs 20ms and runs 1920 times (38.4s total). If it does not need to be rebuilt
  per test, a module or session scope removes most of that.

What the numbers mean

fixtures is setup plus teardown, totalled per fixture across every time it ran. The each column is that total divided by the number of setups, so it includes teardown work too. Teardown is attributed by wrapping finalizers as they are registered, since pytest reports setup time for you but hands teardown to finalizers attached to the fixture definition.

tests is the call phase only. pytest charges fixture setup to the test's setup phase, so adding the setup phase and the fixture totals together would count the same seconds twice.

collection is the time between pytest starting collection and finishing it, which is where your test modules get imported.

unattributed is whatever the run spent that none of the buckets explain: interpreter startup, pytest's own import, reporting, plugin overhead. It is reported rather than hidden, because a large value there is itself worth knowing.

imports is measured separately, in its own --collect-only run under -X importtime, with a bare interpreter's startup subtracted. It covers pytest and its plugins as well as your modules, so it overlaps the breakdown rather than being another slice of it. That is why it sits on its own line. Skip it with --no-imports if you only want the session numbers.

Fixtures with the same name in different files are tracked separately, so a client fixture defined in three conftests does not appear as one confusing total.

Budgets in CI

[tool.testcost]
pytest_args = ["tests"]
max_total_seconds = 300
max_collect_seconds = 10
max_import_seconds = 5

testcost check

fail 6m02.1s total, 19.8s collecting, 24.1s importing, 1284 tests
     total took 6m02.1s, over the 5m00.0s budget by 1m02.1s
     imports took 24.1s, over the 5.00s budget by 19.1s

     heaviest fixtures: postgres_container (1m52.0s), db_transaction (38.4s), seeded_catalogue (10.6s)

Exits non-zero when a budget is blown, and names the heaviest fixtures so the next step is obvious.

Configuration

All keys live under [tool.testcost] in pyproject.toml. All are optional.

Command reference

Both commands exit non-zero if pytest itself did, so a CI step cannot go green after the suite went red. The profile is still printed, because it is still valid.

How it compares

Notes

Times come from one run, so a suite with genuinely variable timing needs more than one look. There is no averaging across runs yet.

The plugin is installed as a pytest entry point, which means it is imported by every pytest process on the machine. Without --testcost-report or TESTCOST_REPORT it registers nothing and every hook returns immediately.

Fixture teardown attribution wraps finalizers on the fixture definition. If another plugin replaces the finalizer list wholesale after setup, that fixture's teardown time will be missing rather than wrong.

Fixtures are identified by the file that defines them rather than the module name, because every standalone conftest.py imports as conftest and three of them defining client would otherwise collapse into one row belonging to none of them.

Running under -n with pytest-xdist is not supported yet: timings from several worker processes are not merged.

Contributing

Bug reports and pull requests are welcome. uv sync then uv run pytest to get started.

License

MIT.