typemut

Mutation testing for Python type annotations.

Standard mutation testing tools (cosmic-ray, mutmut) mutate runtime code and check if tests catch it. typemut mutates only type annotations and checks if type checkers (mypy, pyright) catch the change.

• Mutant killed = type checker reports an error (types are strict enough)
• Mutant survived = no type error (types are too loose or type checker coverage is weak)

Installation

pip install typemut
# or
uv add typemut

Quick Start

1. Create typemut.toml in your project root:

[typemut]
module-path = "src/myproject"
test-command = "make typecheck"  # must exit non-zero on type errors
checker = "mypy"  # or pyright/basedpyright; needed when test-command is a wrapper like make
timeout = 30

[typemut.operators]
remove-union-member = true
remove-literal-member = true
widen-type = true
remove-optional = true
add-optional = true
widen-container-type = true
swap-iterator-generator = true
typevar-variance = true

2. Run:

typemut run                          # full pipeline: discover + execute + report
typemut html --open                  # generate HTML report and open in browser

Or from another directory:

typemut -C /path/to/project run

3. Run on several workers. Each mutant is a full type checker run, so a single worker is slow on a real project. --jobs N runs mutants on N workers in parallel:

git status --porcelain               # must print nothing: workers only see committed code
typemut run --jobs 8                 # 8 type checker processes at once

With --jobs greater than 1, typemut creates N temporary git worktrees of the current commit, splits the mutants between them (all mutants of one file go to the same worker), runs each worker in its own process and removes the worktrees at the end. What this means for your project:

• The working tree must be clean. Uncommitted or untracked files would not be in the worktrees, so typemut refuses to start and lists them. Commit or stash them first.
• test-command runs inside each worktree, not in your project directory. Keep module-path and paths in test-command relative to the project root. Git-ignored files (.venv, generated code) are not copied: the type checker must be callable without them — e.g. installed in an activated virtualenv or on PATH.
• The type checker's cache is copied into each worktree, so workers start warm: the baseline check fills it in the project first, then every new worktree gets a copy with modification times and symlinks kept (mypy drops cache entries whose times differ). For mypy that is the directory shown as cache: in the Type checker: line (see Type checkers), copied only when it is configured as a relative path inside the project, under the configured name (a symlinked .mypy_cache lands in the worktree as .mypy_cache, where mypy looks for it). mypy in a worktree finds an absolute cache directory (MYPY_CACHE_DIR=/abs/path, an absolute --cache-dir or cache_dir, or one built from ~ or $VAR) at the same place as in the project, so all workers share that one cache instead (a warning says so); use a relative path inside the project to give each worker its own copy. A cache outside the project (../mypy-cache) is neither listed nor copied, and workers start cold; --cache-dir=/dev/null turns mypy's cache off, so there is nothing to copy or warn about. pyright and generic have no cache to copy. A missing cache is skipped and a failed copy is logged as a warning; that worker then starts cold. If the copy is interrupted (Ctrl-C), the half-made worktree is removed.
• Pick N by CPU cores and memory: every worker runs its own type checker process (mypy on a large project can take gigabytes of RAM). The number of CPU cores is a good start.
• The baseline check (type checker on unmodified code) runs once, in the project itself, before the workers start.

--jobs is also available on typemut exec. Without it (or with --jobs 1) mutants run one by one directly in your project, which needs neither git nor a clean tree.

Commands

Global option: -C, --project-dir PATH — change to this directory before running (the target project's root, where typemut.toml lives).

init + exec + report are the steps of run. Running them separately lets you resume: results are saved as they arrive, so after an interrupted exec (Ctrl+C, CI timeout) run typemut exec again and it continues with the mutants that are still pending.

typemut run

typemut init

typemut exec

typemut report

typemut html

Using in CI

typemut run and typemut report exit with code 0 regardless of survivors unless you ask them to gate:

typemut run --fail-under 90                          # exit 1 if the mutation score is below 90%
typemut run --baseline typemut-baseline.json         # exit 1 on survivors not in the baseline
typemut run --baseline typemut-baseline.json --update-baseline   # accept the current survivors

A baseline is a JSON file of survived mutants you have reviewed and accept (e.g. equivalent mutants the type checker can never kill). Commit it and run with --baseline in CI: known survivors pass, new ones fail the build and are listed. Entries are matched by file, operator, original and mutated annotation, and the text of the source line — not its number, so edits elsewhere in a file keep the baseline valid. Entries that no longer survive are reported; rerun with --update-baseline to drop them.

run and report also exit with code 1 while any mutant is still pending (a run stopped by --max-duration, an interrupted exec), and then do not update the baseline: a gate on incomplete results would pass by accident.

Mutation testing is slow, so use --jobs N in CI (it needs a clean git working tree, see Quick Start), and --incremental with the database kept between CI runs (below). typemut runs this on itself — see make mutate and .github/workflows/ci.yml.

Fast CI: incremental runs

typemut run --incremental reuses the kills of earlier runs stored in the same database, and runs only the mutants that are new, survived, ended in error, or whose kill may no longer hold:

typemut run --incremental --jobs 4 --baseline typemut-baseline.json

What is reused. A killed mutant is identified by its file, line, column, operator, original and mutated annotation and required import. When it is killed, typemut records the files the kill depends on, with a hash of each: the mutated file, the files the checker reported errors in, and every project file those import (directly or not). The next incremental run copies the kill (status and checker output) if all those files are unchanged, and prints Reused N cached kills.

What always runs again. Survived and error mutants; kills by timeout; kills whose errors are outside the project root (in installed packages) or whose output names no file (a test-command that cds before running the checker); every mutant under the generic checker, which cannot tell the files a kill depends on (the run says so, and stores nothing in the cache); kills whose files changed (the run prints how many). The baseline check always runs too: it catches a broken tree and warms the checker's cache.

When the whole cache is dropped. The cache is reset, with the changed input named (Result cache reset: checker version changed.), when any of these differ from the run that filled it: the typemut version (and, for unreleased builds, its sources), the checker and its version, the version of the Python it runs with, test-command, timeout, the checker's config files (mypy.ini, pyproject.toml, pyrightconfig.json, ...) and the files matched by cache-key-files (by default the lockfiles uv.lock, poetry.lock, requirements*.txt). Patterns in cache-key-files are relative to the project root; absolute, empty and invalid ones are skipped with a warning. If the checker's version is unknown (see Type checkers; set checker-version-command), that run neither reuses nor stores results and leaves the cache as it is, so a single failed version check does not discard it: the next run with a known version reuses the kills again. The Python version is that of python --version, run through the project runner the checker runs with (uv run python --version for uv run mypy, python3.12 --version for python3.12 -m mypy) or from the directory of a checker run by path (.venv/bin/python --version for .venv/bin/mypy).

Keeping the cache fresh. typemut run --refresh runs every mutant and stores the results without reusing any: run it on a schedule (e.g. weekly) to recheck the kills. (exec --refresh only runs the mutants still pending.) Under --max-duration a refresh rechecks only the mutants that fit: they run in a fixed order, so the ones past the limit are never rechecked and keep their old kills. Give the scheduled refresh a time limit (and CI timeout) long enough for the whole project, or none. Without --incremental nothing is reused or stored, and the kills already in the result cache stay there: a run without the flag (or init) between two incremental runs does not discard them.

Time limits. --max-duration SECONDS (or max-duration in the config) stops starting new mutants after that many seconds, waits for the running ones and saves every finished result; the run then fails with N mutants not run yet. Keep it below the CI job timeout. A run stopped by SIGTERM (as CI runners stop jobs) saves its finished results too. On a project with no cache yet, a first run may not fit in the CI timeout: store the database after every run on the main branch (even a failed one), and each next incremental run continues from the kills of the previous ones until the whole project is covered.

The cache lives in the database file (typemut.sqlite by default): keep it out of git (it is binary and changes on every run; with --jobs an untracked file in the project fails the clean-tree check, so list it in .gitignore or pass --db outside the project) and carry it between CI runs as a build artifact or cache (see below).

Limits. Reuse trusts the checker's output and typemut's import graph. For every imported project module both its .py file and its .pyi stub are tracked, so adding a stub next to a module (or a module next to a stub) reruns the kills that depend on it. A file the checker reads that is not imported (a plugin config, a stub found through mypy_path outside the project, a new module that shadows another on the search path) is not tracked; list it in cache-key-files if a change to it can turn a kill into a survivor. For a checker run by name without a project runner (mypy src, npx pyright), the Python version is that of the python on PATH, which may not be the one the checker runs with (a pipx-installed mypy), or may be missing (systems with only python3): then a Python upgrade does not reset the cache; run the checker through its environment (uv run mypy, .venv/bin/mypy) or use --refresh.

Keeping the database between CI runs

An incremental run is only as fast as the database it starts from, and a CI job starts from a fresh checkout. Carry typemut.sqlite from run to run with these rules:

• Not in git. The database is binary, changes on every run, and an untracked or modified file in the project fails the clean-tree check of --jobs. Keep *.sqlite in .gitignore (or pass --db outside the project).
• Only trusted runs on the default branch store it (push, schedule, manual runs). Pull/merge requests and other branches only read it: a cached kill is copied as is, so a database written by an untrusted change could hide survivors from later runs.
• Every run, the default branch included, starts from the latest database of the default branch, whatever that run's outcome. A run that failed (a new survivor) or was stopped by --max-duration saved its finished results, and the next run continues from them. That is how a project too large for one CI job gets covered: each run on main adds kills until none are left to run.
• No database means a full run: correct, only slower. An expired artifact or an evicted cache costs one slow run, nothing else.
• Time limits nest: --max-duration < the step's (script's) timeout < the job's timeout, so typemut stops starting mutants, waits for the running ones and saves, and the upload step still runs. Leave room for one mutant's timeout plus the baseline check between --max-duration and the step timeout.
• Runs on the default branch wait for each other. Two overlapping runs would both upload a database, and the older commit's run could finish last and become "the latest". Queue them (a concurrency group on GitHub, a resource_group on GitLab), so a queued run downloads the database of the run before it when it starts, and pick the newest run's database, not the newest upload (the artifact examples do; the GitHub cache example restores the newest upload, see its caveat). A queued run waits for the one ahead of it, so a push that arrives during the weekly refresh waits until the refresh is done.
• Recheck on a schedule: a weekly typemut run --refresh on the default branch runs every mutant again and stores the results like a push. It rechecks only what fits in its --max-duration, so give it a limit (and job timeout) long enough for the whole project, within the platform's maximum job time: the GitHub examples use 5.5 hours, the GitLab one 5h40m, instead of 25 minutes.

typemut's own CI does this with a GitHub artifact: see .github/workflows/ci.yml. (There, any change to typemut's own sources resets the cache, since an unreleased typemut is keyed by its sources: pull requests that touch src/typemut/ run every mutant.)

GitHub Actions: artifact

Artifacts are kept for retention-days (up to the repository's limit, 90 days by default), can be found by name through the API, and a failed or timed-out run still uploads one with if: always(). The download step looks for the unexpired typemut-db artifact of the newest run (highest run id) on main of this repository: head_repository_id keeps out a fork's pull request from a branch that is also named main, and the run id keeps out a re-run of an old run, which uploads later than newer runs. It reads the 100 most recently uploaded typemut-db artifacts, which is plenty with one per run on main. The concurrency group queues runs of the same branch, so a run on main starts after the previous one has uploaded. GitHub keeps only one waiting run per group, and a newer one cancels it. For a waiting push run that is fine: the newer run downloads the same database and covers its commit. But a push can also cancel a waiting weekly refresh (or wait for a running one, up to its time limit); re-trigger a cancelled refresh by hand with the workflow's refresh input. Don't give refreshes a group of their own: a push run started during a refresh has a higher run id, so its database, built without the refresh's results, would become the latest. overwrite: true lets a re-run of failed jobs, which reuses the run, replace that run's artifact. The weekly --refresh run gets a 5.5-hour limit (a GitHub-hosted job runs for at most 6 hours).

name: typemut

on:
  push:
    branches: [main]
  pull_request:
  schedule:
    - cron: "0 3 * * 1"   # weekly --refresh
  workflow_dispatch:
    inputs:
      refresh:            # re-trigger a refresh replaced in the queue
        type: boolean
        default: false

permissions:
  contents: read

jobs:
  typemut:
    runs-on: ubuntu-latest
    # > step timeout > --max-duration; the weekly --refresh gets longer.
    timeout-minutes: ${{ (github.event_name == 'schedule' || inputs.refresh) && 350 || 45 }}
    concurrency:                   # runs on main upload one after another
      group: typemut-db-${{ github.ref }}
      cancel-in-progress: false
    permissions:
      contents: read
      actions: read                # list and download artifacts
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v4
      - run: uv sync

      - name: Download the typemut database of main
        env:
          GH_TOKEN: ${{ github.token }}
          REPO_ID: ${{ github.repository_id }}
        run: |
          # Unexpired artifact of the newest run on main of this repository
          # (not of a fork), whatever that run's outcome.
          query='[.artifacts[]
                  | select(.expired | not)
                  | select(.workflow_run.head_branch == "main")
                  | select(.workflow_run.head_repository_id == ($ENV.REPO_ID | tonumber))]
                 | max_by(.workflow_run.id) // empty
                 | "\(.id) \(.workflow_run.id)"'
          if ! found="$(gh api "repos/$GITHUB_REPOSITORY/actions/artifacts?name=typemut-db&per_page=100" --jq "$query")"; then
            echo "::warning::Could not list the typemut-db artifacts: running every mutant"
            exit 0
          fi
          if [ -z "$found" ]; then
            echo "::notice::No typemut database of main: running every mutant"
            exit 0
          fi
          read -r artifact_id run_id <<< "$found"
          if gh api "repos/$GITHUB_REPOSITORY/actions/artifacts/$artifact_id/zip" > typemut-db.zip \
              && unzip -o -q typemut-db.zip typemut.sqlite; then
            echo "::notice::typemut database from run $run_id"
          else
            echo "::warning::Could not download the database of run $run_id: running every mutant"
            rm -f typemut.sqlite
          fi
          rm -f typemut-db.zip

      - name: typemut
        timeout-minutes: ${{ (github.event_name == 'schedule' || inputs.refresh) && 340 || 35 }}
        env:
          ARGS: ${{ (github.event_name == 'schedule' || inputs.refresh) && '--refresh --max-duration 19800' || '--max-duration 1500' }}
        run: >-
          uv run typemut run --incremental --jobs 4
          --baseline typemut-baseline.json $ARGS

      # Only trusted events on main store the database, even when typemut failed.
      - name: Upload the typemut database
        if: always() && github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
        uses: actions/upload-artifact@v4
        with:
          name: typemut-db
          path: typemut.sqlite
          retention-days: 30
          if-no-files-found: ignore
          overwrite: true          # a re-run of failed jobs reuses the run

GitHub Actions: cache

Simpler, with no API calls, but caches unused for 7 days are evicted and the repository's cache storage is limited (10 GB by default, oldest evicted first). Each run on main saves a new cache under a unique key; every run restores the newest one by the typemut- prefix. Pull requests can restore caches of the base branch, and caches saved by pull requests stay scoped to them, so main never reads one. The triggers, time limits and concurrency group are those of the artifact workflow; only the database steps differ (no actions: read needed).

The caveat: the restore step takes the newest saved cache, not the newest run's. The concurrency group does not stop a re-run of an older run on main (say, "Re-run failed jobs" on a run that failed): it saves last, so later runs restore it instead of the newer runs' caches. Nothing wrong gets reused, since kills are only copied for files whose hashes still match (and a changed typemut, checker version or lockfile resets the cache), but the progress of the runs in between is lost and their mutants run again. Re-run only the newest run on main, or use the artifact variant, which picks the newest run.

name: typemut

on:
  push:
    branches: [main]
  pull_request:
  schedule:
    - cron: "0 3 * * 1"   # weekly --refresh
  workflow_dispatch:
    inputs:
      refresh:            # re-trigger a refresh replaced in the queue
        type: boolean
        default: false

permissions:
  contents: read

jobs:
  typemut:
    runs-on: ubuntu-latest
    # > step timeout > --max-duration; the weekly --refresh gets longer.
    timeout-minutes: ${{ (github.event_name == 'schedule' || inputs.refresh) && 350 || 45 }}
    concurrency:                   # runs on main save one after another
      group: typemut-db-${{ github.ref }}
      cancel-in-progress: false
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v4
      - run: uv sync

      - uses: actions/cache/restore@v4
        with:
          path: typemut.sqlite
          key: typemut-${{ github.run_id }}-${{ github.run_attempt }}
          restore-keys: typemut-   # the newest cache saved on main

      - name: typemut
        timeout-minutes: ${{ (github.event_name == 'schedule' || inputs.refresh) && 340 || 35 }}
        env:
          ARGS: ${{ (github.event_name == 'schedule' || inputs.refresh) && '--refresh --max-duration 19800' || '--max-duration 1500' }}
        run: >-
          uv run typemut run --incremental --jobs 4
          --baseline typemut-baseline.json $ARGS

      - uses: actions/cache/save@v4
        if: always() && github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
        with:
          path: typemut.sqlite
          key: typemut-${{ github.run_id }}-${{ github.run_attempt }}

GitLab CI: artifact

Three jobs share the script: typemut runs on the default branch and keeps the database as an artifact even when it fails (when: always); typemut-mr runs for merge requests and keeps nothing; typemut-refresh is the weekly --refresh from a pipeline schedule, with a longer time limit so that it rechecks the whole project (raise the project's and the runner's maximum job timeout to match). All download the artifact of the typemut or typemut-refresh job of the newest default-branch pipeline in which that job succeeded or failed: GitLab's "latest artifact of a branch" endpoint only knows successful pipelines, so the job is looked up through the API, in the 50 newest pipelines of the default branch (listed by the server, so merge request pipelines do not push them out). A pipeline still running other jobs (a later stage, a deploy waiting on a manual job) counts once its typemut job is done; pipelines are ordered by id, so a retried job of an older pipeline does not win. With none among them, the run is a full one. resource_group: typemut-db is GitLab's counterpart to the GitHub concurrency group: the typemut and typemut-refresh jobs run one at a time, and a job waiting for the group downloads the database when its script starts, after the job ahead of it has uploaded. A push pipeline that starts during the weekly refresh therefore waits for it (up to its 6h timeout). The group's default process mode lets waiting jobs start in any order; set it to oldest_first (curl --request PUT --header "PRIVATE-TOKEN: ..." --data "process_mode=oldest_first" "$CI_API_V4_URL/projects/<id>/resource_groups/typemut-db") so they run in pipeline order and the newest pipeline's database is also the last one written. Listing pipelines and jobs needs a token with the read_api scope (a project access token with the Reporter role, saved as the masked CI/CD variable TYPEMUT_API_TOKEN; leave it unprotected so merge request pipelines see it). Without it the script falls back to the job token and the typemut job of the latest successful pipeline. curl --fail fetches the single file, so no zip is involved; a missing artifact (or a failed download, whose partial file is removed) just means a full run. RUNNER_SCRIPT_TIMEOUT below timeout leaves the runner time to upload the artifact after a script timeout; it needs GitLab Runner 16.4 or later (older runners ignore it, and a job that hits timeout uploads nothing).

.typemut:
  image: ghcr.io/astral-sh/uv:python3.13-bookworm
  timeout: 45m
  variables:
    RUNNER_SCRIPT_TIMEOUT: 35m     # < timeout, > --max-duration (Runner 16.4+)
    TYPEMUT_MAX_DURATION: "1500"
  script:
    - uv sync
    - |
      api="$CI_API_V4_URL/projects/$CI_PROJECT_ID"
      if [ -n "${TYPEMUT_API_TOKEN:-}" ]; then
        # Newest `typemut`/`typemut-refresh` job on the default branch,
        # successful or failed, in its 50 newest pipelines.
        job_id="$(python3 - <<'PY'
      import json, os, urllib.parse, urllib.request
      api = os.environ["CI_API_V4_URL"] + "/projects/" + os.environ["CI_PROJECT_ID"]
      def get(path):
          headers = {"PRIVATE-TOKEN": os.environ["TYPEMUT_API_TOKEN"]}
          with urllib.request.urlopen(urllib.request.Request(api + path, headers=headers)) as response:
              return json.load(response)
      ref = urllib.parse.quote(os.environ["CI_DEFAULT_BRANCH"], safe="")
      for pipeline in get(f"/pipelines?ref={ref}&order_by=id&sort=desc&per_page=50"):
          jobs = get(f"/pipelines/{pipeline['id']}/jobs?scope[]=success&scope[]=failed&per_page=100")
          found = [job["id"] for job in jobs if job["name"] in ("typemut", "typemut-refresh")]
          if found:
              print(found[0])
              break
      PY
      )" || job_id=""
        if [ -n "$job_id" ]; then
          curl --fail --location -sS --header "PRIVATE-TOKEN: $TYPEMUT_API_TOKEN" \
            "$api/jobs/$job_id/artifacts/typemut.sqlite" -o typemut.sqlite \
            && echo "typemut database from job $job_id" \
            || { echo "no previous DB"; rm -f typemut.sqlite; }
        else
          echo "no previous DB"
        fi
      else
        # Latest successful pipeline of the default branch only.
        curl --fail --location -sS --header "JOB-TOKEN: $CI_JOB_TOKEN" \
          "$api/jobs/artifacts/$CI_DEFAULT_BRANCH/raw/typemut.sqlite?job=typemut" -o typemut.sqlite \
          || { echo "no previous DB"; rm -f typemut.sqlite; }
      fi
    - uv run typemut run --incremental --jobs 4 --max-duration $TYPEMUT_MAX_DURATION
        --baseline typemut-baseline.json $TYPEMUT_EXTRA_ARGS

.typemut-store:
  extends: .typemut
  resource_group: typemut-db       # one job at a time, like GitHub's concurrency group
  artifacts:
    paths: [typemut.sqlite]
    when: always                   # failed and timed-out runs store their results too
    expire_in: 30 days

typemut:
  extends: .typemut-store
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

typemut-refresh:                   # weekly pipeline schedule on the default branch
  extends: .typemut-store
  timeout: 6h
  variables:
    RUNNER_SCRIPT_TIMEOUT: 5h50m
    TYPEMUT_MAX_DURATION: "20400"  # 5h40m: a refresh rechecks only what fits
    TYPEMUT_EXTRA_ARGS: "--refresh"
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

typemut-mr:
  extends: .typemut
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

GitLab CI: cache

Works with caveats, so prefer artifacts:

• By default protected branches (the default branch) and unprotected ones (merge request branches) get separate caches, so a merge request never sees the cache of the default branch. Turning off Use separate caches for protected branches fixes that, but then any branch's job can write the cache the default branch reads, unless every job but the default branch's uses policy: pull.
• fallback_keys needs GitLab 16.0 or later, a variable in cache:policy 16.1 or later, $CI_DEFAULT_BRANCH_SLUG (the fallback key must be slugged like the key the default branch saves under) 17.4 or later, and RUNNER_SCRIPT_TIMEOUT GitLab Runner 16.4 or later.
• Without a distributed cache (S3, GCS, ...), each runner has its own cache, and a job on another runner starts from nothing.

typemut:
  image: ghcr.io/astral-sh/uv:python3.13-bookworm
  timeout: 45m
  variables:
    RUNNER_SCRIPT_TIMEOUT: 35m
  cache:
    key: typemut-db-$CI_COMMIT_REF_SLUG
    fallback_keys: [typemut-db-$CI_DEFAULT_BRANCH_SLUG]
    paths: [typemut.sqlite]
    when: always
    policy: $TYPEMUT_CACHE_POLICY
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      variables:
        TYPEMUT_CACHE_POLICY: pull-push
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      variables:
        TYPEMUT_CACHE_POLICY: pull   # merge requests only read the cache
  script:
    - uv sync
    - uv run typemut run --incremental --jobs 4 --max-duration 1500 --baseline typemut-baseline.json

Type checkers

typemut runs test-command as is, but it reads the output with a plugin for the type checker that command runs. The plugin tells a real kill from a mutant that merely broke the code: a failure whose only errors are a missing name, a syntax error or an invalid type is recorded as error, not killed. It also finds the checker's config files and cache directory. The checker's version, the files with errors and which project files a file imports are known to the plugin too: incremental runs use them to tell whether an earlier kill can be reused.

By default (checker = "auto") the checker is detected from test-command: a recognised executable counts when it is the command itself, possibly after environment assignments, env/exec/time, or a runner (uv run, poetry run, pipx run, pdm run, hatch run, rye run, pnpm exec, uvx, npx, bunx, python -m), in any command of a chain (uv run mypy src, python -m mypy -p app, cd src && npx pyright). Runner options are skipped, with their values (uv run --group dev mypy, npx -p pyright pyright). An executable name passed to another tool (tox -e mypy, pre-commit run mypy --all-files) is not detected. A command that runs none of them (make typecheck, a script, a wrapper like tox), or more than one (mypy src && pyright src), gets the generic checker and a warning; name the checker instead:

[typemut]
test-command = "make typecheck"
checker = "mypy"

The run prints the checker it uses, with the config files it found and the cache directory it would use (and, if configured as a relative path inside the project, copy into each worktree with --jobs), relative to the project root (Type checker: mypy (config: pyproject.toml; cache: .mypy_cache)); the cache is listed even before mypy has created it. mypy's cache directory comes from --cache-dir, MYPY_CACHE_DIR, cache_dir in its config, or .mypy_cache, with ~ and $VAR expanded as mypy does. An unknown checker name is logged as a warning and generic is used. mypy's --pretty, --show-column-numbers and --show-error-end output is understood; only errors count (mypy notes, pyright warnings and information are ignored). Both stdout and stderr are parsed, so noise from wrappers such as uv run on stderr does not hide the checker's errors.

Incremental runs reuse earlier kills only while the checker's version is the same, and do not use the cache at all while it is unknown. It is taken from test-command up to the checker executable plus --version, as written there (cd backend && uv run mypy . -> cd backend && uv run mypy --version: of the commands before the checker only cd and pushd are kept). If the command does not run the executable directly (tox -e mypy, make typecheck), set checker-version-command, e.g. checker-version-command = "npx pyright --version"; the first X.Y.Z in its output is the version. generic only has a version when checker-version-command is set, but it never reuses kills anyway: it cannot tell which files they depend on.

Third-party checkers

A package can ship a checker plugin: subclass typemut.checkers.base.TypeChecker and register it under the typemut.checkers entry point group. The entry point name is the value for checker = ...:

# pyproject.toml of the plugin package
[project.entry-points."typemut.checkers"]
pytype = "typemut_pytype:PytypeChecker"

from collections.abc import Iterable

from typemut.checkers.base import Diagnostic, TypeChecker


class PytypeChecker(TypeChecker):
    name = "pytype"
    executables = frozenset({"pytype"})

    def parse_output(self, output: str) -> Iterable[Diagnostic]:
        ...  # one Diagnostic(path, code) per error

Set name, executables (used for detection and the version command), false_kill_codes and config_names (config files the checker reads from the project root), and override parse_output(output) to return a Diagnostic(path, code) (from typemut.checkers.base) for every error in one output stream. executables should not overlap those of another checker: checker = "auto" falls back to generic when a command matches two unrelated checkers. The one exception is a subclass of a checker that shares its executables (e.g. class StrictMypyChecker(MypyChecker) with the inherited mypy): auto-detection then picks the subclass instead of its base. Two such subclasses of the same checker installed together match both, so set checker = "<name>" explicitly. config_files, cache_paths (cache directories inside the project), worktree_cache_paths (those copied into each worktree; by default all of cache_paths, mypy leaves out a cache configured by an absolute path), python_version and dependencies (by default: the project files reachable through imports; None means that kill is not reused) can be overridden too. A checker that can never tell which files a kill depends on (e.g. its output carries no file paths) should set traces_dependencies = False: incremental runs then neither reuse nor store its results and leave the cache alone, as with generic. The constructor takes (test_command, version_command, runner, graph=None, environ=None); environ is the environment the checker runs with (the process environment by default). An entry point that fails to import or does not name a TypeChecker subclass is logged as a warning and skipped.

What It Finds

typemut generates mutations of type annotations and checks whether the type checker catches them. Each mutation operator targets a specific class of type safety issues.

RemoveUnionMember

Removes one member from a union type.

# Original
def handle(value: int | str | float) -> None: ...

# Mutant: remove str
def handle(value: int | float) -> None: ...

Survived = your code doesn't distinguish between union members. If removing str from the union causes no type error, it means no code path relies on value being a str. The union may be overly broad, or the type checker doesn't see the code that handles str specifically.

RemoveOptional

Removes None from X | None.

# Original
def find_user(id: int) -> User | None: ...

# Mutant
def find_user(id: int) -> User: ...

Survived = callers don't check for None. The return type says "might be None" but no consumer's type annotations actually require a None-check. Either the None case is dead code, or callers use # type: ignore.

AddOptional

Adds | None to return types and class fields (parameters are excluded — callers simply won't pass None, making those mutations uninformative).

# Original
class Config:
    name: str

# Mutant
class Config:
    name: str | None

Survived = consumers don't rely on non-None guarantee. The field claims to always have a value, but no typed code would break if it could be None. This often reveals missing type coverage in code that reads the field.

WidenType

Replaces a concrete class with its parent (base) class to find places where a more abstract type could be used.

class Animal: ...
class Cat(Animal): ...
class Dog(Animal): ...

# Original
def feed(pet: Cat) -> None: ...

# Mutant
def feed(pet: Animal) -> None: ...

Survived = the code doesn't rely on the concrete subclass. The function could accept the broader base type, suggesting the annotation is more specific than necessary.

RemoveLiteralMember

Removes one value from a multi-value Literal[...].

# Original
def open_file(mode: Literal["r", "w", "a"]) -> None: ...

# Mutant: remove "a"
def open_file(mode: Literal["r", "w"]) -> None: ...

Survived = that literal value is never used. No typed caller passes "a", so the Literal is broader than the code actually needs.

WidenContainerType

Replaces a concrete container with the next more abstract type.

# Original
def total(items: list[int]) -> int: ...

# Mutant
def total(items: Sequence[int]) -> int: ...

Widening steps: list/tuple → Sequence, set/frozenset → AbstractSet, dict → Mapping, Sequence/AbstractSet/Mapping → Collection, Collection → Iterable. tuple[X, ...] becomes Sequence[X], Mapping[K, V] becomes Collection[K], and heterogeneous tuples like tuple[int, str] are skipped.

Survived = the code only uses the abstract interface. A parameter could accept the wider type; a return type could promise less.

SwapIteratorGenerator

Widens iterator/generator types by one step: Iterator → Iterable, Generator[Y, S, R] → Iterator[Y], and the async equivalents.

# Original
def read_lines(path: str) -> Generator[str, None, None]: ...

# Mutant
def read_lines(path: str) -> Iterator[str]: ...

Survived = nobody uses the extra capability. Callers never call next() / send() or read the generator's return value, so the simpler type is enough.

TypeVarVariance

Removes covariant=True / contravariant=True from a TypeVar, or adds either one to an invariant TypeVar.

# Original
T_co = TypeVar("T_co", covariant=True)

# Mutant
T_co = TypeVar("T_co")

Survived = variance is not relied on (removal), or the TypeVar could be declared variant (addition).

Plugins

Core operators know nothing about libraries. Some libraries wrap your types in their own generics and read them at runtime, which changes what a mutation means. Plugins teach typemut these rules. They are opt-in:

[typemut]
plugins = ["sqlalchemy", "fastapi"]

An unknown plugin name is logged as a warning and skipped. A plugin can claim an annotation and decide its mutations (usually by running the core operators on a wrapped type and dropping mutations that are invalid for the library), and can contribute its own operators. Annotations no plugin claims are mutated as usual.

Third-party plugins

A package can ship its own plugin: subclass typemut.plugins.base.Plugin and register it under the typemut.plugins entry point group. The entry point name is the name used in plugins = [...]:

# pyproject.toml of the plugin package
[project.entry-points."typemut.plugins"]
django = "typemut_django:DjangoPlugin"

An entry point that fails to import or does not name a Plugin subclass is logged as a warning and skipped.

A plugin's find_mutations receives the enabled operators and the class registry. The Plugin base class also provides parso helpers (_node_code, _trailer_target) that a plugin can use or override. Operators a plugin returns from operators() subclass TypeMutationOperator, which provides _node_code, _is_qualified_name, _union_members and _type_params. The data types plugins and operators work with (AnnotationNode, AnnotationContext, Mutation) live in typemut.model. A plugin operator's config_key can be used in [typemut.operators] and [typemut.ignore-types] like a built-in operator key. Keys that no available operator has are reported as a warning and have no effect.

sqlalchemy

For SQLAlchemy 2.0 declarative models (Mapped[...], also orm.Mapped[...]). SQLAlchemy derives the column type, nullability and relationship target from the type inside Mapped[...], so the plugin mutates that type instead of the whole annotation:

Rules:

• Quoted forward references (Mapped["UserDB | None"]) are mutated inside the quotes.
• WidenContainerType skips the top-level type inside Mapped[...]: SQLAlchemy needs a concrete column type / collection class there (list, set), not an ABC. Nested containers (Mapped[dict[str, list[int]]]) are still widened.
• On relationship attributes (= relationship(...)):
  • WidenType is skipped: the target must be a mapped class, and widening usually points at the declarative base;
  • AddOptional is skipped for collections (Mapped[list[X]]): a collection relationship is an empty collection, never None.
• WriteOnlyMapped[...] and DynamicMapped[...] get no mutations: they are always relationship collections, and none of the core mutations is valid for them.

Nullability mismatches between Mapped[X | None] and mapped_column(nullable=...) are a runtime/schema issue, not a typing one, so the plugin does not generate them.

fastapi

FastAPI calls route handlers itself, and tests reach them over HTTP, so no Python code calls them. The type checker has no caller to check a mutated signature against: -> RedirectResponse → -> RedirectResponse | None always survives. The plugin skips all parameter and return annotations of such functions:

from fastapi import APIRouter

router = APIRouter()

@router.get("/login")
async def login(next: str | None = None) -> RedirectResponse:  # not mutated
    target: str | None = next  # still mutated
    ...

A function counts as an endpoint when it is decorated with @<receiver>.<method>(...), where <method> is a route method (get, post, put, patch, delete, head, options, trace, api_route, websocket) or exception_handler, middleware, on_event. The receiver has to be an instance of fastapi.FastAPI / fastapi.APIRouter, or of a project class inheriting from them (also across modules). The type is resolved statically from imports, not from variable names:

• app = FastAPI(), router: APIRouter = make_router(), def register(app: FastAPI);
• app = FastAPI() inside an app factory function;
• a receiver imported from another project module, including re-exports (from app.api import router, from app import api + @api.router.get).

Unresolvable receivers (a factory call without annotation, self.router, a star import) are not treated as endpoints. The same goes for functions registered without a decorator (app.add_api_route(...)) or used only through Depends(...). Mark those with # pragma: no mutate.

Other frameworks: [typemut.entrypoints]

Other frameworks register handlers the same way: in-house RPC layers, message consumers, task queues. Name them in [typemut.entrypoints] to skip the parameter and return annotations of their handlers, without a plugin per framework (no entry in plugins needed):

[typemut.entrypoints]
# functions decorated with one of these
decorators = ["mylib.rpc.*.method", "faststream.*.subscriber", "taskiq.*.task"]
# methods of classes inheriting from one of these
base-classes = ["mylib.rpc.Endpoint"]
# functions with a parameter whose annotation references one of these (DI markers)
parameter-types = ["mylib.rpc.*.Dependency", "dishka.FromDishka"]

from mylib.rpc import Endpoint, Router

rpc = Router()


@rpc.method()  # mylib.rpc.Router.method
async def get_item(item_id: int, service: rpc.Dependency[ItemService]) -> ItemSchema:  # not mutated
    item: ItemSchema | None = await service.get(item_id)  # still mutated
    ...


class Items(Endpoint):
    async def get_item(self, item_id: int) -> ItemSchema: ...  # not mutated

Patterns are fnmatch globs (* matches dots too) against qualified names, resolved statically like the fastapi plugin does (imports, aliases, relative imports, re-exports and objects imported from other project modules):

• decorators: @x, @x.y or @x.y(...). The decorator matches as the qualified name of what it refers to (from taskiq import task → taskiq.task), and, when its receiver is an instance (rpc = Router(), rpc: Router = make(), def register(rpc: Router)), as <class>.<attribute> for the instance's class and each of its base classes: @rpc.method() → mylib.rpc.Router.method, also with rpc = AppRouter() where class AppRouter(Router) is a project class. Classes are named as they are imported (from mylib.rpc import Router → mylib.rpc.Router), not where they are defined.
• base-classes: methods (not nested functions) of a class whose bases inherit from a matching class, directly or through project classes in other modules. Methods of the matching class itself are mutated.
• parameter-types: every name the annotation references counts (Dependency[Service], Annotated[Service, Inject]), resolved like decorators: rpc.Dependency with rpc = Router() is mylib.rpc.Router.Dependency. String annotations and default values (= Depends(...)) are not looked at.
• Unresolvable names (factory calls, self.router, star imports) don't match.
• Unknown keys and invalid values are logged as warnings and skipped.

Filtering

Annotations are automatically skipped when:

• The line contains # type: ignore or # pragma: no mutate
• The annotation is Any (mutations are meaningless — Any absorbs all types)
• AddOptional targets a function parameter (low signal — callers won't pass None)

Ignoring library types per operator

By default every operator mutates every annotation, including library wrapper types it knows nothing about (Mapped[int] → Mapped[int] | None). To silence such noise, list the types an operator must not touch under [typemut.ignore-types], keyed by operator (same keys as [typemut.operators]):

[typemut.ignore-types]
all = ["django.db.models.*"]                        # every operator
add-optional = ["sqlalchemy.orm.Mapped"]
remove-optional = ["sqlalchemy.orm.Mapped"]
widen-container-type = ["sqlalchemy.orm.Mapped"]
widen-type = ["sqlalchemy.orm.DeclarativeBase"]

• An operator skips every annotation that references one of its ignored types (anywhere in it, e.g. list[Mapped[int]]); other operators still mutate it.
• An operator also drops mutations that would introduce an ignored type, e.g. WidenType widening a model to DeclarativeBase.
• Patterns are fnmatch globs matched against the qualified name the type is imported under in the file (from sqlalchemy.orm import Mapped → sqlalchemy.orm.Mapped), not where it is defined. Aliases (import sqlalchemy.orm as orm, orm.Mapped) and relative imports of your own packages are resolved.
• Names that aren't imported (builtins, classes defined in the same file) are never ignored.
• Plugins see annotations first and are not affected: with plugins = ["sqlalchemy"], Mapped[...] is still mutated correctly (Mapped[int] → Mapped[int | None]).
• Unknown operator keys and invalid values are logged as warnings and skipped.

Ignoring fields of class hierarchies per operator

Some fields are equivalent mutants by design: on response or serialization models nobody reads the fields in typed code, so AddOptional, WidenType and WidenContainerType on them always survive, while RemoveOptional and RemoveUnionMember still catch real mismatches. List the classes whose fields an operator must not touch under [typemut.ignore-classes], keyed by operator (same keys as [typemut.operators]):

[typemut.ignore-classes]
add-optional = ["myproject.api.BaseResponse"]
widen-type = ["myproject.api.BaseResponse"]
widen-container-type = ["myproject.api.BaseResponse"]
all = []                                            # every operator

• An operator skips the fields (class-level annotations, name: str) of every class that is, or inherits from, one of its ignored classes, directly or through any number of project classes, in any module. Other operators still mutate them.
• Parameters, return types and local variables of the methods of those classes are mutated as usual.
• Patterns are fnmatch globs matched against qualified names: the class's own (myproject.api.ItemResponse; nested classes as module.Outer.Inner) and those of all its bases. A base is named the way it is imported (from pydantic import BaseModel → pydantic.BaseModel), a project class also by the module it is defined in, so a class re-exported from a package's __init__.py matches both myproject.api.BaseResponse and myproject.api.base.BaseResponse.
• Plugins see annotations first and are not affected.
• Unknown operator keys and invalid values are logged as warnings and skipped.

Config Reference

[typemut]
module-path = "src/myproject"           # directory to scan for annotations
test-command = "make typecheck"         # command to run type checker
timeout = 30                            # seconds per mutation
excluded-modules = ["src/vendor/*.py"]  # glob patterns to skip
skip-comments = ["type: ignore", "pragma: no mutate"]
db = "typemut.sqlite"                   # database file
plugins = ["sqlalchemy"]                # library plugins, none by default
checker = "mypy"                        # mypy, pyright, basedpyright, generic; default "auto" detects it
                                        # from test-command (set it when test-command is a wrapper like make)
# checker-version-command = "mypy --version"  # optional; for incremental runs;
                                        # default: test-command up to the checker + --version
incremental = false                     # reuse kills of earlier runs (--incremental / --no-incremental)
# max-duration = 1500                   # seconds; start no new mutant after it (--max-duration)
cache-key-files = ["uv.lock", "poetry.lock", "requirements*.txt"]  # glob patterns relative to the
                                        # project root (absolute, empty or invalid ones are skipped with a warning);
                                        # a change to any drops every cached result

[typemut.operators]
# all enabled by default, disable selectively
remove-union-member = true
remove-literal-member = true
widen-type = true
remove-optional = true
add-optional = true
widen-container-type = true
swap-iterator-generator = true
typevar-variance = true

[typemut.ignore-types]
# qualified type patterns each operator must not touch; "all" applies to every operator
all = []
add-optional = ["sqlalchemy.orm.Mapped"]

[typemut.ignore-classes]
# qualified class patterns whose fields (and their subclasses' fields) each operator
# must not touch; "all" applies to every operator
all = []
add-optional = ["myproject.api.BaseResponse"]

[typemut.entrypoints]
# qualified name patterns of framework-called handlers whose signatures are not mutated
decorators = []                         # functions decorated with these
base-classes = []                       # methods of classes inheriting from these
parameter-types = []                    # functions with a parameter annotated with these

HTML Report

The HTML report shows:

• Summary stats and per-module mutation scores
• Each mutant as a collapsible card with unified diff
• Color-coded status: killed (green), survived (red), error (orange)
• Full type checker output per mutant
• Expand/Collapse All controls

typemut html --open                     # save and open in browser
typemut html -o report.html             # save to specific file

Development

make install    # create venv and install with dev deps
make test       # run tests
make lint       # run ruff, flake8/wps, mypy
make mutate     # run typemut on itself (--incremental: reuses kills in typemut.sqlite);
                # fails on survivors missing from typemut-baseline.json
                # extra options: make mutate MUTATE_ARGS="--refresh"
make mutate-baseline  # accept the current survivors (review the diff before committing)

Dependencies

• parso — CST parsing (preserves formatting and whitespace)
• rich — terminal reporting
• click — CLI framework