quality-gate

A ratchet quality gate. It reads what your linters, type checkers, test runners and scanners already wrote, compares every metric to a committed baseline, and fails the build if any of them got worse.

Not a threshold gate. There is no number you have to reach before it is useful — the baseline starts wherever your code is today, and the only rule is that it may not move backwards. Improvements are adopted automatically after merge, so the bar rises on its own and never falls.

## Ratchet Quality Gate

| Package | Metric | Baseline | Current | Δ | Status |
|---|---|---:|---:|---:|:---:|
| `api` | coverage_lines_pct | 89.09 | 88.22 | -0.87 | FAIL |
| `api` | complexity_violations | 108 | 108 | +0 | ok |
| `api` | mypy_errors | 0 | 0 | +0 | ok |
| `(global)` | secret_findings | 0 | 0 | +0 | ok |

**FAIL** — 1 regression(s):
- `api` / **coverage_lines_pct** 89.09 → 88.22

Why this repo exists

Five repos were running five copies of the same 400–1500 line script. They had diverged: different exclusion lists, different report paths, one repo carrying a one-line bug fix that never reached the other four because there was no mechanism for it to travel by. Comparing them, almost every difference was configuration wearing the costume of code.

So the program lives here once, and everything that legitimately differs per repo lives in that repo's quality-gate.toml.

Install

uvx ratchet-gate@0.1.0 check

No install step in CI, no vendored copy to keep in sync, and the pin says exactly which version scored a given run.

The distribution is ratchet-gate; the command is available as both ratchet-gate and quality-gate, so an adopting workflow can keep whichever name it already says.

From a machine on the LAN you can install straight from the source of truth instead, which is what scripts/verify-against-repo.sh does when checking an unreleased commit:

uvx --from git+https://git.marim.dev/mateuscmarim/quality-gate@v0.1.0 quality-gate check

git.marim.dev has no public DNS record, so that form works from self-hosted runners and nowhere else. PyPI is the form to put in a workflow. See RELEASING.md.

Use

quality-gate check                              # score this run, exit 1 on regression
quality-gate check --baseline-ref origin/main   # score against the stricter baseline
quality-gate check --measured api,worker        # only these packages ran
quality-gate promote --monotonic                # adopt improvements only (post-merge)

Point it at a directory of tool output — .quality/<package>/coverage.json, ruff.json, mypy.txt, and so on — and a quality-baseline.json.

Configure

quality-gate.toml at the repo root. Every key is optional; a repo whose layout matches the defaults needs no file at all.

[gate]
artifacts_dir = ".quality"
baseline_path = "quality-baseline.json"
non_ratcheted_dirs = ["tests", "scripts", "notebooks"]
default_baseline_ref = "origin/main"

non_ratcheted_dirs is the one most repos change. Findings in those directories stay in the uploaded artifacts — they just do not gate. One-off analysis scripts, notebook data collectors and test helpers are read top-to-bottom and run by hand; a 200-line main() is the right shape for what they are.

Two things it will not do

It will not let a branch set its own bar. Every number the gate enforces comes out of a tracked file on the branch being scored, so the same commit that adds 47 lint errors can raise lint_errors to 47 and pass. That needs no malice: promote without --monotonic sets baseline = current for every metric, so a branch that runs it locally to seed one new row rewrites all the others. Pass --baseline-ref origin/main in CI and each metric is scored against the stricter of the two baselines.

It will not go green when it could not measure something. A missing gitleaks.json is not a clean secret scan; a crashed bandit is not zero findings; a jscpd report that never got written is not zero duplication. Every one of those is an error that exits 1. All three used to be written the other way round, and each one silently disabled the check it was reporting on.

Supported

Developing

uv venv && uv pip install -e '.[dev]'
.venv/bin/python -m pytest        # includes the equivalence suite
.venv/bin/python -m mypy          # --strict, src and tests

The suite runs the implementation this package replaces side by side with this one over identical inputs and asserts their stdout, stderr, exit code and written files match exactly. That is what lets a repo switch over without its gate changing verdict on the day it does. Every intended departure is listed in DIVERGENCES.md; anything else is a bug.

To check against a repo's real reports rather than synthetic ones:

scripts/verify-against-repo.sh ../../trainwithme/nasa nasa-server

Design notes live in the Quality Gate project on mddocs.