riskratchet
A maintainability ratchet for AI-assisted Python and TypeScript. The bar can only move down.
AI coding agents are very good at writing code that compiles, runs, and passes the tests they ship with it. They are less good at:
- writing meaningful tests for the new code,
- noticing a 30-line function quietly became 130 lines,
- catching that the public API now exposes a function with no callers in tests,
- realising a small refactor turned an
ifladder into a 14-way cyclomatic monster.
A traditional review catches some of this. A ratchet catches all of it, mechanically, every time. riskratchet computes a per-function risk score from coverage gaps, cyclomatic complexity, churn, public surface, and sprawl, then fails CI or blocks the commit whenever risk grows past a baseline. Nobody has to play complexity cop.
The review workflow is inspired by
cargo-crap (which made the CRAP
metric practical in CI with baselines, PR comments, and JSON output) and
Cursor's thermo-nuclear-code-quality-review
agent prompt (which emphasises maintainability, structure, sprawl, and
explicit boundaries). riskratchet is neither a Python port of cargo-crap nor an
agent prompt: it reports CRAP and adds Python-specific signals on top (branch
gaps, churn, public surface, sprawl).
Quickstart
pip install riskratchet
# or run without installing
uvx riskratchet --help
# 1. run your tests with coverage in JSON form
pytest --cov --cov-report=json:coverage.json
# 2. snapshot the current risk profile
riskratchet baseline src --coverage coverage.json --output .riskratchet.json
# 3. inspect what was captured
riskratchet scan src --coverage coverage.json
# 4. fail the build when risk regresses
riskratchet check src --coverage coverage.json --baseline .riskratchet.json
riskratchet check exits 1 on regressions, 2 on usage errors (e.g. missing
baseline), and 0 otherwise.
For early adoption before a baseline exists, check --fail-above N gates
on an absolute threshold without requiring a baseline (baseline gating
remains the recommended mode for mature codebases):
# No baseline yet: fail if any function scores above 60.
riskratchet check src --coverage coverage.json --fail-above 60
# scan also exposes a no-baseline gate (different exit/output shape).
riskratchet scan src --coverage coverage.json --fail-above 75
riskratchet scan src --coverage coverage.json --fail-severity high
When --baseline and --fail-above are both given, the baseline gate
is authoritative and --fail-above is ignored with a stderr warning.
Setting up riskratchet
riskratchet init scaffolds a [tool.riskratchet] section in
pyproject.toml and prints a ready-to-paste CI snippet. With
--with-baseline (or by saying yes to the interactive prompt on a
TTY when pytest is detected), it also runs pytest --cov and creates
the baseline in one go:
riskratchet init # write config, print snippet
riskratchet init --with-baseline # also run pytest --cov + baseline
riskratchet init --force # replace existing [tool.riskratchet]
riskratchet doctor is a pre-flight that names whatever would make
check fail to start (missing paths, missing/malformed baseline,
missing/stale coverage, no git history, unknown config keys, invalid
suppressions) and prints the exact fix command for each. Since 0.3.7 it
also reports scoring-model: what would score this run, and whether the
baseline was scored the same way. With
typescript = true it also checks the [typescript] extra is installed
and reports the TypeScript coverage report (ts-coverage: missing or
malformed is a FAIL, none configured is a WARN); on a TypeScript-only
tree the Python coverage row reads "not applicable", the same rule
check applies. The status table goes to stdout; the → fix:
remediations go to stderr so you can pipe them separately:
riskratchet doctor # human-readable table + remediation
riskratchet doctor --json # validates against schemas/doctor.schema.json
riskratchet doctor 2>/dev/null # status table only
riskratchet doctor >/dev/null # remediation commands only
doctor exits 0 only when every check is pass or warn; a single
fail exits 1. The check names in --json are a closed enum in
doctor.schema.json, so a new row is a schema change — pin a copy and you
must refresh it when one is added. The intended workflow is init → doctor → fix the
warnings → baseline → check.
GitHub Action
The composite action ships in action.yml so adopters don't have to
copy a workflow file — uses: KayhanB21/riskratchet@v0.3.9 is the
canonical reference. The action installs riskratchet via uv tool install, runs check (--format pr-comment in both baseline and
no-baseline modes), upserts a sticky PR comment, and surfaces the
check exit status so PR checks reflect regressions.
# .github/workflows/riskratchet.yml
on: [pull_request]
jobs:
riskratchet:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
# Full history: churn uses `git log --since`, which on the default
# shallow (depth-1) clone sees only HEAD and silently scores every
# function's churn as zero — so CI would disagree with your baseline.
fetch-depth: 0
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'
# Install your project and its test dependencies however you normally do:
- run: pip install -e '.[dev]'
- run: pytest --cov --cov-branch --cov-report=json:coverage.json -q
- uses: KayhanB21/riskratchet@v0.3.9
with:
coverage: coverage.json
The two steps before the action are not optional. coverage: coverage.json names a
report, and a path you name must exist — riskratchet exits 2 rather than scoring
every function as uncovered. Leaving the input out does not help either: auto-coverage
shells out to pytest, which is not on the PATH of the uv tool install environment
the action creates, so that is exit 2 as well. Whatever writes coverage.json is
yours to choose — coverage run -m unittest discover && coverage json -o coverage.json
works just as well — as long as the workflow writes it before the action runs.
Inputs (defaults in parentheses): paths ([tool.riskratchet] paths), coverage (auto-detected), baseline (.riskratchet.json
— when the file is missing, the action runs in --fail-above mode),
fail-above (60), comment (true), python-version (3.12),
riskratchet-version (latest from PyPI), github-token
(${{ github.token }}), and, since 0.3.6, typescript (empty:
[tool.riskratchet] typescript decides; true / false pass
--typescript / --no-typescript), ts-coverage and ts-entry
(space-separated, workspace-relative; each entry becomes one
--ts-coverage / --ts-entry). The action installs the [typescript]
extra on every path, so turning TypeScript on from config works in CI
without further setup — at v0.3.6 or later: an older action.yml
that installs the latest CLI from PyPI gets 0.3.6 without the extra, and
a typescript = true config then exits 2 with the install hint. See
TypeScript for a full CI example.
For Marketplace discovery, the KayhanB21/riskratchet-action
wrapper repo is the recommended entry point; it delegates to the
root action.yml so both shapes share one source of truth.
Verifying releases
Every tagged release ships supply-chain provenance you can inspect: a CycloneDX
SBOM of the wheel's runtime dependency closure (the sbom workflow artifact), a
signed GitHub build-provenance attestation on the wheel and sdist, and PEP 740
PyPI attestations from Trusted Publishing. To confirm a downloaded wheel was built
by this repo:
gh attestation verify riskratchet-<version>-py3-none-any.whl --owner KayhanB21
See docs/threat-model.md for
what each artifact does and does not vouch for.
The canonical use case: AI agent + side project
You've been vibe-coding a FastAPI backend with an AI agent for eight months.
It works, tests are green-ish (62% coverage), but you just noticed
services/billing.py::reconcile_subscriptions quietly grew to 180 lines and
an 11-way match statement you don't remember writing.
pip install riskratchet
pytest --cov --cov-branch --cov-report=json:coverage.json
riskratchet scan src --coverage coverage.json --top 10
reconcile_subscriptions shows up at score 71 (high) with
structural_complexity: 90, sprawl: 55, coverage_gap: 60. You also spot a
surprise: a 12-line public utility _normalize_plan_id scoring 48 because it
has zero tests. Snapshot the bar:
riskratchet baseline src --coverage coverage.json --output .riskratchet.json
git add .riskratchet.json && git commit -m "Add riskratchet baseline"
From here, every time the agent adds a webhook handler or "refactors the
billing flow," run riskratchet check before committing. If it quietly bloated
reconcile_subscriptions from 180 to 220 lines, the check exits 1 and names
the regression. You stop having to remember to look.
Why this is the canonical use case: AI agents are excellent at adding code, mediocre at noticing they've made things worse. The baseline is your memory.
Other patterns
- Team gating PRs in CI. Run
pytest --covandriskratchet check --format pr-commentin GitHub Actions; pipe togh pr comment. The PR-comment format starts with<!-- riskratchet-report -->so the bot updates the same comment on each push instead of spamming. The ratchet is mechanical and unowned, so nobody has to play "the complexity cop" in code review. See Using riskratchet from an AI coding agent. - Pre-commit hook for a solo repo. Wire
pytest-covandriskratchetinto.pre-commit-config.yamlso every commit regenerates coverage and gates the commit on no regressions. See Pre-commit integration. - Investigating one ugly function. Use
riskratchet explain path/to/file.py::qualnameto dump the six component scores and find the driver (complexity vs. coverage vs. sprawl). After refactoring, runriskratchet diff --json | jq '.improved[], .regressed[]'to prove the change was net-positive, not just rearranging deck chairs.
Why CRAP alone is useful but incomplete
The classic CRAP score (CC^2 * (1 - line_coverage)^3 + CC) catches one
shape of bad code: complex and poorly tested. That's a real problem, but
it misses several others that ship to production just as often:
- A function with low complexity and zero tests. CRAP gives it
CC(a single digit). Risk is real but invisible. - A function with full line coverage but every branch covered the same way. CRAP only looks at line coverage.
- A function in a 2,000-line module everyone is afraid to touch. Sprawl is invisible to CRAP.
- A function that changed in 40 of the last 90 commits. Churn is invisible to CRAP.
riskratchet keeps CRAP as a reported metric and computes its own composite score from six weighted components so those other risks show up too.
Pre-commit integration
How pre-commit and riskratchet fit together
Two things about pre-commit matter for riskratchet:
- Pre-commit hides your unstaged edits before running hooks. Hooks only see the code you're actually about to commit. Useful in general, but it means riskratchet sees a different source tree than the one open in your editor.
- Each
language: pythonhook runs in its own isolated virtualenv that contains riskratchet and its declared deps, not your project's pytest, application code, or test plugins.
Together these create one requirement: the coverage.json riskratchet reads
must reflect the same stashed source tree it's analyzing. Reusing an old
coverage.json from before pre-commit stashed your edits drifts source and
coverage out of sync.
That's why the published hook ships with --no-auto-cov --allow-missing-coverage by default: safe but limited. Pick one of the
patterns below to make it useful.
Pattern A: pre-generate coverage in a sibling hook (recommended)
Run pytest --cov inside the same pre-commit chain so the coverage matches
the stashed tree exactly.
repos:
- repo: local
hooks:
- id: pytest-cov
name: pytest --cov (produces coverage.json for riskratchet)
entry: pytest --cov --cov-branch --cov-report=json:coverage.json -q
language: system
pass_filenames: false
always_run: true
- repo: https://github.com/KayhanB21/riskratchet
rev: v0.3.9
hooks:
- id: riskratchet
args:
- "src"
- "--coverage"
- "coverage.json"
- "--baseline"
- ".riskratchet.json"
Variant: a TypeScript (or mixed) repo
Add the report your JavaScript runner writes as a sibling hook and hand it to
riskratchet. Run riskratchet from your own environment (language: system) with the
[typescript] extra installed there — uv add --dev 'riskratchet[typescript]' or
pip install 'riskratchet[typescript]' — since the published hook's isolated venv
does not carry the extra. typescript = true in [tool.riskratchet] (or
--typescript on the command) turns the backend on:
repos:
- repo: local
hooks:
- id: vitest-cov
name: vitest --coverage (produces coverage/lcov.info for riskratchet)
entry: npx vitest run --coverage --coverage.reporter=lcov
language: system
pass_filenames: false
always_run: true
- id: riskratchet
entry: riskratchet check src --ts-coverage coverage/lcov.info --baseline .riskratchet.json
language: system
pass_filenames: false
always_run: true
Variant: uv / poetry projects (all language: system)
Skip the isolated venv entirely and run both hooks inside your project's environment. This is what riskratchet itself uses:
repos:
- repo: local
hooks:
- id: pytest-cov
entry: uv run pytest --cov --cov-branch --cov-report=json:coverage.json -q
language: system
pass_filenames: false
always_run: true
- id: riskratchet
entry: uv run riskratchet check src --coverage coverage.json --baseline .riskratchet.json --no-auto-cov
language: system
pass_filenames: false
always_run: true
Two upsides: single env for both hooks (no isolated-venv surprises), and
uv run resolves the same Python and deps uv sync set up. Downside:
contributors must have uv installed locally.
Pattern B: let riskratchet run pytest itself
Override the hook to language: system so it inherits your shell PATH (and
finds your real pytest):
repos:
- repo: local
hooks:
- id: riskratchet
entry: riskratchet check src --baseline .riskratchet.json
language: system
pass_filenames: false
always_run: true
riskratchet runs the configured [tool.riskratchet] test_command (default
pytest --cov --cov-branch --cov-report=json:{output} -q) and caches the
result under .riskratchet/coverage.json. The cache is reused until any .py
file under the scan paths is newer.
For local development outside pre-commit, auto-coverage applies to plain
riskratchet scan|baseline|check too; pass --no-auto-cov to opt out.
Using riskratchet from an AI coding agent
riskratchet is designed to be called from agents and parsed without
screen-scraping. See AGENTS.md for the full operational
contract; the recipes below cover the common cases.
Top three highest-risk functions:
riskratchet scan src --coverage coverage.json --json \
| jq '.functions[:3] | .[] | {qualname, score, severity}'
Full baseline diff including improvements and removed functions:
riskratchet diff src --coverage coverage.json \
--baseline .riskratchet.json --json
Gate a CI job on regressions:
riskratchet check src --coverage coverage.json \
--baseline .riskratchet.json --json > regressions.json
status=$?
if [ "$status" -eq 1 ]; then
jq -r '.regressions[] | "- \(.qualname): \(.reason)"' regressions.json
exit 1
fi
exit "$status"
Post regressions as a PR comment (use --format pr-comment for a sticky body
that updates in place via the <!-- riskratchet-report --> marker; use
--format github for inline workflow warnings):
Changed in 0.3.5.
check --format pr-commentnow renders the set the gate acted on, in both baseline and--fail-abovemode, so the comment can never contradict the exit code printed beside it. In baseline mode the diff rides along underneath as collapsed context. Previously the baseline-mode comment selected rows by diff status, which could report "No risk regressions detected" on a run that exited 1, and show a regression row on a run that exited 0.
riskratchet check src --coverage coverage.json \
--baseline .riskratchet.json --format markdown \
| gh pr comment --body-file -
Markdown and PR-comment output can link each row back to source:
riskratchet scan src --format pr-comment \
--repo-url https://github.com/acme/project \
--commit-ref "$GITHUB_SHA"
In GitHub Actions, those values are filled from GITHUB_SERVER_URL,
GITHUB_REPOSITORY, and GITHUB_SHA when available.
JSON output is validated against the schemas under
schemas/ on every release. All nine:
report.schema.json:scan --jsonregressions.schema.json:check --jsondiff.schema.json:diff --jsonbaseline.schema.json:.riskratchet.jsonon disksummary.schema.json:scan|check|diff --summary --jsonconfig.schema.json:config show --jsondoctor.schema.json:doctor --jsonexplain.schema.json:explain --jsondebug.schema.json:--debug-json
Native JSON output includes $schema and version fields so consumers can pin
parsing behavior.
Getting the schemas. Since 0.3.7 they ship inside the wheel and the sdist, so the package you installed carries its own contract:
from riskratchet.schemas import schema_names, schema_path, load_schema
schema_names() # ('baseline', 'config', 'debug', ... ) — all nine
load_schema("report") # the parsed schema, ready for a validator
schema_path("report") # its path on disk, for tools that want the file
Every $schema and $id is a URL that resolves, pointing at the copy on master
(https://raw.githubusercontent.com/KayhanB21/riskratchet/master/schemas/<name>.schema.json).
Before 0.3.7 they pointed at a path that 404s and shipped in no artifact at all.
What version means, per document. Three different things wear the name, so
read it against the document you are holding:
| Document | version is |
|---|---|
scan / check / diff / explain / --summary JSON |
the output contract version, riskratchet's own MAJOR.MINOR ("0.3") |
config show --json, doctor --json |
the package version ("0.3.9") |
--debug-json |
an integer contract version of its own, independent of both |
.riskratchet.json |
the baseline format version ("3"), independent of all three |
The output-contract version derives from the package's minor, which is exactly the
guarantee already in force: field names are stable within a minor version, and renames
or removals travel under a Breaking heading. It read "0.2" from 0.2.x through
0.3.6 — including straight across 0.3.0's Breaking output change — because it was
maintained by hand.
The schemas are strict. Every one sets additionalProperties: false, and
doctor.schema.json closes its check-name enum. That is what makes them worth pinning
— an unexpected key is an error, not a shrug — but it also means any additive
field requires a pinned copy to be refreshed, in a patch release as much as a minor
one. The version field tells you which shape you have; the CHANGELOG lists additions
under Changed with "refresh your pinned copy" whenever a strict schema grew.
Common pitfalls
- Running
checkwithout a baseline.riskratchet baselinemust run first (typically onmain) and the resulting.riskratchet.jsonchecked in. Exits2when missing. - Passing
coverage.xmlto--coverage. riskratchet readscoverage.json. Generate it withpytest --cov --cov-report=json:coverage.json. - Parsing stdout as both prose and JSON. Pick a format. With
--json, stdout is a single JSON object; status messages go to stderr. Whencheckexits1, a short hint with the two escape hatches (regenerate baseline, or loosen the per-component gate) is written to stderr, so stdout stays clean. - Bumping the baseline to silence a regression. The baseline is the bar;
if it has to move up, do it in a dedicated PR with a written justification.
In
checkoutput, "new" means absent from the baseline, so a function added in an earlier commit can still appear as new until the baseline intentionally accepts it.
For the broader trust boundaries and non-goals, see
docs/threat-model.md.
Suppressions and partial coverage
--exclude skips files at discovery time. --allow analyzes a file but
suppresses matching functions from reporting and gating:
riskratchet check src --baseline .riskratchet.json \
--allow "GeneratedModel.*" \
--allow "src/generated/**"
Function patterns match dotted qualified names. Patterns containing / or
** match repo-relative POSIX paths.
If an allow pattern suppresses a function that the baseline holds, check
reports the entry as left the gate, not deleted. For more information, see
Baseline format.
The default missing-coverage policy is pessimistic: unmapped functions are treated as uncovered. For partial local runs:
riskratchet scan src --coverage coverage.json --missing-coverage optimistic
riskratchet scan src --coverage coverage.json --missing-coverage skip
optimistic treats missing file coverage as fully covered. skip drops
functions from unmapped files and reports the skipped count.
A scan path outside the config directory (riskratchet scan ../shared/src) is keyed by a
../ path relative to it — one spelling per file whatever the cwd, since 0.3.6 — and the
command says so once on stderr, because a baseline written against such a key is only
matched by runs that resolve the same config directory.
Generated files. A file whose header carries a comment-anchored @generated
marker (# @generated in Python, // @generated or /* @generated */ in
TypeScript) is skipped in both languages since 0.3.6: its functions are not
scored or gated, the file still appears in files[] with zero functions, and the
summary says so (skipped_generated_files, "N generated files skipped"). A
marker inside a string, a docstring, or trailing code does not count. TypeScript
also skips by name (*.pb.ts, *.gen.ts). Python codegen that says DO NOT EDIT without the marker — protobuf stubs, for one — is scored like any other
Python, because those headers come in a hundred spellings and riskratchet does
not guess. Keep them out at discovery time instead:
[tool.riskratchet]
exclude = ["**/*_pb2.py", "**/*_pb2_grpc.py", "src/generated/**"]
Pytest plugin
riskratchet ships a pytest plugin that runs check as part of your test
session:
pytest \
--cov --cov-report=json:coverage.json \
--riskratchet \
--riskratchet-paths src \
--riskratchet-baseline .riskratchet.json
The session exits non-zero when riskratchet finds regressions, so CI can gate
on pytest alone.
The plugin reads [tool.riskratchet], so it gates on the same policy
riskratchet check does: scan paths, thresholds, weights,
include/exclude/allow, churn window, and the missing-coverage policy all
come from your config, and its output is redacted when you have asked for that.
A flag below overrides config for that one setting; leave it off and config
wins. It also refuses to pass a session whose scan found nothing while the
baseline holds entries, and warns on a shallow clone — the same guards the CLI
has.
Changed in 0.3.8. These were the last two options whose default was a real path rather than "unset", so
[tool.riskratchet] baselineandcoveragecould not be read at all:riskratchet checkgated against the configured file while the plugin gated against its own literal — and the missing-baseline message told you to runriskratchet baseline, which writes to the configured path, so following it could never help. A flag still beats config; a config value anchors to the config directory, an explicit flag to the pytest rootdir.Also in 0.3.8: the per-file warnings the plugin raises while scanning now redact when you have asked for that, so
private_comment = truecovers the whole session's output rather than the regressions table alone.
Changed in 0.3.5. Before this the plugin read no config at all. A repo with
paths = ["lib"]had it scanning a non-existentsrc; a repo that had tightenedfail_regression_aboveto1still got its hardcoded5; customweightswere ignored, so its scores could not be compared with the baseline the CLI wrote; andprivate_comment = truedid not stop it printing raw paths into CI logs. Expect it to start agreeing withriskratchet check— including failing where it used to pass. (The last of those was only half fixed: the regressions table redacted from 0.3.5, but the warnings raised while scanning kept naming real modules until 0.3.8.)
Available flags (each defaults to the [tool.riskratchet] value, then to the
value shown):
--riskratchet(required to enable)--riskratchet-paths(default:paths, elsesrc; repeatable)--riskratchet-baseline(default:baseline, else.riskratchet.json; since 0.3.8)--riskratchet-coverage(default:coverage, elsecoverage.json; since 0.3.8)--riskratchet-fail-new-above(default:50)--riskratchet-fail-regression-above(default:5)--riskratchet-fail-existing-above(default: unset)--riskratchet-fail-component-regression-above(default:15)--riskratchet-no-component-regression-gate--riskratchet-typescript/--riskratchet-no-typescript(default:typescript, else off; since 0.3.6)--riskratchet-ts-coverage(default:ts_coverage; repeatable; since 0.3.6)--riskratchet-ts-entry(default:ts_entry; repeatable; since 0.3.6)--riskratchet-no-redact-paths,--riskratchet-no-redact-qualnames,--riskratchet-no-private-comment,--riskratchet-no-allow-missing-coverage(since 0.3.7) — the plugin has no positive form for these config-only settings, but a setting config can turn on must be one a flag can turn back off at every door
Since 0.3.6 the plugin scores TypeScript exactly as riskratchet check does — same
[typescript] extra, same rule for a missing report (the session fails unless
allow_missing_coverage), same warning when the baseline holds TypeScript entries the run did
not analyze. Every setup failure fails the session with the message, never a traceback.
How risk is scored
Each function gets six component scores in [0, 100]:
| Component | Weight | What it measures |
|---|---|---|
| coverage_gap | 30% | 1 - line_coverage |
| structural_complexity | 25% | cyclomatic complexity, saturating at CC=20 |
| branch_gap | 15% | 1 - branch_coverage when branch coverage is known |
| churn | 10% | commits in the last 90 days, saturating at 10 |
| public_surface | 10% | coverage gap penalised harder when the function is public |
| sprawl | 10% | function length and file length blended |
Total risk is the weighted sum. Severity bands: 0-24 low, 25-49 medium, 50-74 high, 75-100 critical.
is_public is determined statically from the AST: by qualname when no
__all__ is declared (leading-underscore is private, dunders are public);
by additive promotion from a static __all__ (omission never demotes);
fall back to the naming rule when __all__ is dynamic. Full rules in
AGENTS.md.
Components, in plain English
Each component is rescaled to [0, 100] (where 100 = maximum risk for that
signal) before being weighted into the total. Here's what each one actually
means, with a concrete example.
coverage_gap: "is this function tested at all?"
The fraction of lines in the function that your test suite never executes.
A function with 100% line coverage scores 0; a function with 0% line
coverage scores 100.
Example: a 40-line
parse_invoicewhere your tests only exercise the happy path (28 lines covered, 12 missed) givescoverage_gap = 30. A brand-newmigrate_to_v2with no tests at all givescoverage_gap = 100.
structural_complexity: "how many ways can this function go?"
Cyclomatic complexity, which roughly counts independent paths through the
function (each if, elif, and, or, for, except adds one).
Saturates at CC=20; anything past that is already "very complex" and
there's no value in keeping count.
Example: a getter with one return statement is
CC=1, score 0. Avalidate_user_inputwith 6 chainedif/elifbranches isCC=7, score ~35. A 14-waymatchstatement isCC=15, score ~75.
branch_gap: "are both sides of every if tested?"
Like coverage_gap, but for branches. A function whose tests only ever
take the if True path of an if/else will have full line coverage but
only 50% branch coverage. Only counts when your coverage run included
--cov-branch.
Example:
def discount(user): return 0.2 if user.is_premium else 0.0. A test that only passes premium users gets 100% line coverage but 50% branch coverage, sobranch_gap = 50.
churn: "how often does this function change?"
Number of git commits touching the function's line range in the configured
churn window (default 90 days, set with --churn-days or [tool.riskratchet] churn_window_days). Saturates at 10 commits. High churn means many people
have edited it recently, which correlates with bugs.
Example: a stable
parse_iso_datelast touched two years ago ischurn = 0. Apricing_engine.calculate_totaledited in 14 of the last 90 commits saturates at 10, sochurn = 100.
Churn is read from the configuration directory, which may not be your repository
root. If [tool.riskratchet] lives in services/api/pyproject.toml, riskratchet looks
for history in services/api — and a git repository keeps its history at the top level,
so it finds none. Every function then scores churn = 0: not because nothing changed, but
because nothing was read.
Since 0.3.7 riskratchet says so, on every command and in doctor's git row. It does not
yet change the number. Scoring churn from the repository root would raise the churn
component on nearly every function in such a project at once, and a patch release must not
turn a green gate red — so 0.4.0 makes that change, deliberately and with a release
note. Until then, run from the repository root if you want churn scored, or pass
--no-git to opt out and silence the warning.
The same applies to every other way churn can come back empty. A shallow clone, a git that
is not on the PATH, a timeout, a fork that fails under memory pressure: each used to
produce churn = 0 in silence, indistinguishable from "this code is stable", and then got
written into the next baseline as though it were a measurement. All of them now warn, and
none of them is a gate failure — the scores are missing a component, which is not the same
as the code being worse.
public_surface: "if this breaks, do callers we can't see break too?"
A multiplier on coverage gap: when a function is part of your public API,
its missing coverage is penalised harder than the same gap on a private
helper. A private helper with 40% coverage is a problem you can fix
locally; a public function with 40% coverage is a contract problem.
Example:
_normalize_pathwith 50% coverage givespublic_surface = 25. Publicformat_currencywith 50% coverage givespublic_surface = 50._LegacyExposedlisted in__all__with 50% coverage givespublic_surface = 50(promoted to public despite the underscore).
sprawl: "is this function (or its file) just too big?"
A blend of function length and the surrounding file's length. Long
functions are harder to hold in your head; long files mean any function in
them has more neighbors competing for attention. Both contribute.
Example: a 12-line function in a 200-line file gives
sprawl = 5. A 180-line function in a 2,000-line module givessprawl = 85.
A worked example
Suppose services/billing.py::reconcile_subscriptions is 180 lines, public,
has CC=14, 55% line coverage, 40% branch coverage, no recent churn, and
lives in a 900-line file. Its components might look like:
| Component | Raw signal | Score | Weight | Contribution |
|---|---|---|---|---|
| coverage_gap | 45% uncovered | 45 | 0.30 | 13.5 |
| structural_complexity | CC=14 of 20 saturating | 70 | 0.25 | 17.5 |
| branch_gap | 60% uncovered branches | 60 | 0.15 | 9.0 |
| churn | 0 commits in 90 days | 0 | 0.10 | 0.0 |
| public_surface | public + 45% gap | 45 | 0.10 | 4.5 |
| sprawl | long function, big file | 65 | 0.10 | 6.5 |
| total | 51.0 |
Score 51 puts this in the high severity band. The dominant drivers are complexity and branch coverage; if you wanted to lower it without rewriting the function, the cheapest path is adding branch tests, not deleting lines.
Configuring weights
Drop a [tool.riskratchet.weights] table in pyproject.toml to override any
subset; the remaining components keep their defaults and the whole vector is
renormalized. For example, to ignore churn entirely and double-weight
coverage:
[tool.riskratchet.weights]
coverage_gap = 0.6
churn = 0.0
Unknown keys and negative values are rejected at startup so a typo cannot silently weaken the score.
Output formats
riskratchet scan src --coverage coverage.json --format table # default
riskratchet scan src --coverage coverage.json --json # shortcut for --format json
riskratchet scan src --coverage coverage.json --format markdown # for PR comments
riskratchet scan src --coverage coverage.json --format sarif # for SARIF consumers
riskratchet scan src --coverage coverage.json --format github # GitHub Actions annotations
riskratchet scan src --coverage coverage.json --format pr-comment
riskratchet scan src --coverage coverage.json --summary # aggregate lines only
riskratchet scan src --coverage coverage.json --summary --json # schema-backed summary envelope
riskratchet scan src --coverage coverage.json --quiet # drops the trailing summary line
riskratchet scan src --coverage coverage.json --min-score 50 # hide lower-risk functions
riskratchet scan src --coverage coverage.json --top 10 # emit only the top N
Group and Language columns (since 0.3.7). Every per-row table — the three
terminal tables, both markdown tables, and the PR comment — carries a Group
column when [tool.riskratchet.groups] actually placed something, and a
Language column when a non-Python function is in the table. A column appears
when it carries information, so an ungrouped Python-only repo gets the output it
got on 0.3.6, byte for byte. Before 0.3.7 the group column existed in exactly
one of those six tables, where it printed a column of ungrouped whether or not
groups were configured. The group and language fields have been in --json
since 0.3.0 and are unchanged.
SARIF intentionally has a narrower contract than native JSON: scan --format sarif emits current findings after the same score filter used for
annotations, while check --format sarif and diff --format sarif emit only
failing regressions. A clean baseline still produces valid SARIF with an
empty results array. This is a deliberate divergence from cargo-crap, which
rejects combining a baseline with SARIF output; riskratchet instead always
emits a schema-valid SARIF 2.1.0 document (empty when there is nothing to
report), so a code-scanning upload never fails just because the gate is green.
Since 0.3.5 that claim is enforced by validating the output against the
normative SARIF 2.1.0 schema in the test suite, not only against a snapshot.
--format markdown reports the number of functions analyzed, alongside the
number emitted, plus the suppressed and skipped counts. Before 0.3.5 it printed
the emitted count under the "Functions analyzed" label, so --top 5 on a
400-function repo read 5 while --json on the same run reported 400.
--format github escapes the annotation message and the file= property with
the escape sets the Actions runner actually reverses for each. Before 0.3.5 the
two were swapped, so every annotation rendered src/m.py%3A%3Ahuge rather than
src/m.py::huge, and a path containing : or , broke property parsing and
the runner dropped the annotation.
Native JSON output (truncated):
{
"$schema": "https://github.com/KayhanB21/riskratchet/schemas/report.schema.json",
"version": "0.2",
"summary": {
"total_functions": 10,
"analyzed_functions": 42,
"emitted_functions": 10,
"total_files": 6,
"coverage_status": "present",
"suppressed_functions": 1,
"skipped_missing_coverage": 0,
"skipped_generated_files": 0,
"by_severity": { "low": 1, "medium": 6, "high": 3, "critical": 0 }
},
"functions": [
{
"path": "src/foo.py",
"qualname": "Foo.bar",
"score": 62.3,
"severity": "high",
"components": {
"coverage_gap": 80.0, "structural_complexity": 55.0,
"branch_gap": 70.0, "churn": 30.0,
"public_surface": 80.0, "sprawl": 10.0
},
"crap": 12.4
}
]
}
Diagnostics and privacy controls
Diagnostics never touch stdout — they go to stderr (or a file), so --json
consumers and pipes stay clean:
riskratchet scan src --verbose # human-readable run diagnostics on stderr
riskratchet scan src --debug-json # same diagnostics as a JSON envelope on stderr
riskratchet scan src --debug-json-file diag.json # ...or written to a file
The --debug-json envelope reports the coverage source (single / map / auto,
including whether the auto-coverage cache was reused or regenerated), git/churn
settings, include/exclude/allow filter effects, the analysis tallies, and (for
check/diff) the resolved baseline. It is validated against
schemas/debug.schema.json and is its own versioned contract.
When redaction is active, the diagnostics surfaces above (banner, --verbose,
--debug-json) hash their paths too, so a --private-comment run does not leak
through diagnostics.
What redacts, and what deliberately doesn't. Two kinds of message reach stderr, and they follow opposite rules:
- Disclosures about the code under analysis redact: the regressions table, the report in every format, the diagnostics surfaces, and — since 0.3.8 — the per-file warnings raised while scanning (a file that fails to parse, a file with no entry in the coverage data), at the CLI door and the pytest-plugin door alike. A redacted warning carries the same digest as the matching row in the report, so you can still tell which file it is about.
- Setup errors addressed to whoever ran the command stay raw: a missing
baseline, an unreadable coverage report, an invalid config key. Hashing the
filename in "baseline file not found:
.riskratchet.json" would leave you no way to act on it.doctorfollows the same rule for the same reason, and takes no redaction flags.
Two limits worth naming, so the contract is checkable rather than aspirational:
- A message about the config file itself is raised before that file has been
read, so it cannot redact.
resolve_redactiongets its settings from the file being reported on. - Two setup notices name directories you configured — "scan path outside the config directory" and the churn root mismatch. Both stay raw under the rule above: each one asks you to change a path you chose, and a hashed directory would leave nothing to act on. The banner still hashes the same scan roots.
For closed-source repos, redaction hashes identifiers in every output format and in the warnings raised alongside them, while leaving the ratchet decision unchanged (redaction runs after baseline matching):
riskratchet check src --coverage coverage.json --redact-paths # hash file paths
riskratchet check src --coverage coverage.json --redact-qualnames # hash function names
riskratchet check src --coverage coverage.json --private-comment # both + drop source links
Each has an off-switch, so a setting turned on in [tool.riskratchet] can be
turned back off for a single run (since 0.3.7):
riskratchet check src --no-redact-paths # show real paths this run
riskratchet check src --no-redact-qualnames # show real qualnames this run
riskratchet check src --no-private-comment # drop the preset, keep the individual keys
--no-private-comment turns off the preset, not the two keys underneath it: a
config that sets redact_paths on its own still redacts paths. In the other
direction an explicit --no-redact-paths beats an active preset, so no
combination of config and flags leaves redaction stuck on. baseline, check
and diff carry --no-allow-missing-coverage on the same rule.
Salt. Hashes are salted, with this precedence: --redact-salt TEXT, then
RISKRATCHET_REDACT_SALT, then [tool.riskratchet] redact_salt. With none set,
the salt is derived from the commit (GITHUB_REPOSITORY@GITHUB_SHA, else
git rev-parse HEAD); riskratchet warns only when there is no salt source at
all, because unsalted hashes over known paths are guessable. So hashes are
stable within a commit (scan/check/diff in one run correlate) and intentionally
unlinkable across commits and repos — set an explicit --redact-salt if you
need a fixed mapping across commits.
The baseline command does not accept redaction flags — the persisted baseline
is the source of truth for future rename matching and is never hashed.
Config validation, groups, and monorepos
Validate project config before relying on it in CI:
riskratchet config validate --config pyproject.toml
riskratchet config show --config pyproject.toml --json
config validate exits 2 for malformed TOML, unknown keys, invalid value
types, or invalid groups.
config show reports every key [tool.riskratchet] accepts, as this build
resolved it. Since 0.3.7 that includes the four redaction settings, which were
the only accepted keys the payload omitted. redact_salt is reported as
"present" or "absent" and never as its value: config show --json lands in
CI logs and bug reports, and a reader holding the salt can reproduce every hash
that redaction exists to prevent. Presence is read from config alone — the
RISKRATCHET_REDACT_SALT env var and the commit-derived fallback belong to a
run, not to the file.
riskratchet finds config by walking upward from the working directory for the
nearest pyproject.toml containing [tool.riskratchet] (the nearest one wins
if several ancestors define it; pass --config to point at a specific file).
Relative config paths (paths, coverage, baseline, the coverage map, the
coverage cache) resolve against that file's directory, and auto-generated
coverage runs from there too, so running from a nested package directory gives
the same result as running from the project root. An explicit --coverage,
positional path arguments, and the no-argument default all stay relative to
your current directory.
The scanning commands treat the two ways a config can be wrong differently:
| behavior | |
|---|---|
An unknown key (fail_new_abvoe = 1) |
warns, run continues |
A known key with an unusable value (fail_new_above = "50") |
exit 2 |
An unknown key may simply come from a config written for a newer riskratchet, so
refusing to run on one would make upgrading riskratchet the only way to downgrade
it. A known key with a wrong-typed value has no such story — and since 0.3.4 it is
a hard error, because the value was previously discarded and the default applied,
so a repo that thought it had tightened its gate had not. Every unusable value is
reported in one run. doctor reports both classes as a config row without
exiting, and config validate remains the strict gate that also fails on a typo:
- run: riskratchet config validate # exit 2 on unknown keys / malformed config
- run: riskratchet check --baseline .riskratchet.json
Roll function-level results up by package or workspace area with
[tool.riskratchet.groups]. Each function is assigned to the longest
matching repo-relative prefix; ungrouped functions are reported as null in
JSON and ungrouped in text or markdown.
[tool.riskratchet.groups]
core = "src/core"
api = ["src/api", "src/public_api"]
For packages/* / services/* layouts where one coverage.json is not
practical, declare a per-prefix coverage map (or pass --coverage-map on the
CLI; longest matching prefix wins):
[tool.riskratchet]
paths = ["packages/alpha", "packages/beta"]
[tool.riskratchet.coverage_map]
"packages/alpha" = "packages/alpha/coverage.json"
"packages/beta" = "packages/beta/coverage.json"
[tool.riskratchet.groups]
alpha = "packages/alpha"
beta = "packages/beta"
One repo-level baseline (recommended for tight coupling) is global; one baseline per package is useful when packages release independently. Every command prints a diagnostic banner to stderr summarizing the resolved root, scan paths, and coverage source.
Configuration reference
Every key [tool.riskratchet] accepts. Unknown keys warn rather than fail, so a typo is
visible without breaking a build; riskratchet config validate is the strict check and
riskratchet doctor reports the same problems as warnings.
| Key | Type | Default | What it does |
|---|---|---|---|
paths |
list[str] | ["."] |
Files/directories to scan. |
include / exclude |
list[str] | [] |
Glob filters over root-relative POSIX paths. (include from config was ignored before 0.3.5 — only the --include flag applied.) |
allow |
list[str] | [] |
Suppress matching functions from reporting and gating. A pattern containing :: matches the full path::qualname target riskratchet prints, one containing / or ** matches the path, anything else matches the qualname. Patterns that suppress nothing warn. |
baseline |
str | .riskratchet.json |
Baseline file path. |
coverage |
str | — | Single coverage.json path. |
coverage_map |
table | — | Per-prefix coverage, e.g. "packages/a" = "a/cov.json". Mutually exclusive with coverage. |
coverage_cache |
str | .riskratchet/coverage.json |
Where auto_coverage writes. |
auto_coverage |
bool | true |
Run test_command to produce coverage when none is fresh. |
test_command |
str | pytest invocation | Command auto_coverage runs; {output} is substituted. |
allow_missing_coverage |
bool | false |
Continue when coverage is absent instead of exiting 2. --no-allow-missing-coverage overrides for one run (since 0.3.7). |
missing_coverage |
str | pessimistic |
Policy for files absent from coverage: pessimistic, optimistic, or skip. |
typescript |
bool | false |
Also analyze and score TypeScript (since 0.3.6; same as --typescript). --no-typescript overrides it. Needs the [typescript] extra. |
ts_coverage |
list[str] | [] |
Istanbul/LCOV report(s) for TypeScript coverage, relative to the config file (same as --ts-coverage). A missing report is exit 2 on baseline/check/diff and a warning on scan/explain. |
ts_entry |
list[str] | [] |
Package entry file(s) that narrow the TypeScript public surface (same as --ts-entry). |
churn_window_days |
int | 90 |
Churn lookback window. |
weights |
table | see below | Per-component weight overrides; non-zero weights are renormalized. |
groups |
table | — | Named path prefixes for monorepo reporting. |
fail_above |
float | — | Fail when any function's score exceeds N. Makes --baseline optional. |
fail_new_above |
float | — | Fail when a function absent from the baseline scores above N. |
fail_regression_above |
float | — | Fail when a function's score grows by more than N. |
fail_existing_above |
float | — | Fail when a function already in the baseline scores above N. |
fail_component_regression_above |
float | 15 |
Fail when a single component grows by more than N. |
component_regression_gate |
bool | true |
On by default. Enables the per-component check above. |
redact_paths / redact_qualnames |
bool | false |
Hash paths/qualnames in output. --no-redact-paths / --no-redact-qualnames override for one run (since 0.3.7). |
private_comment |
bool | false |
Redact the PR comment. --no-private-comment overrides for one run (since 0.3.7). |
redact_salt |
str | — | Salt for the redaction hash, so digests are stable across runs. config show reports it as "present" / "absent", never its value. |
Component gates
component_regression_gate is enabled by default. Beyond the overall-score checks, check
fails when any single risk component (coverage_gap, structural_complexity, branch_gap,
churn, public_surface, sprawl) grows by more than fail_component_regression_above
(default 15) — even if the total score moved little. This catches a function that quietly
loses its tests while getting simpler. Disable per-run with --no-component-regression-gate.
Exit codes
| Code | Meaning |
|---|---|
0 |
Clean — no gate tripped. |
1 |
A gate tripped: check found regressions, scan --fail-above/--fail-severity matched, or doctor reported a FAIL. |
2 |
Setup or usage error: missing/malformed coverage or baseline, a bad flag, or invalid config. |
scan never exits 1 unless you ask it to with --fail-above or --fail-severity.
A baseline riskratchet cannot read is always exit 2, never 0. An empty baseline passes every
gate, so degrading a broken one to "no entries" would silently switch the ratchet off — see
Baseline format.
For the same reason, a [tool.riskratchet] key whose value this build cannot use is exit 2 rather
than a silently-applied default — see Config validation.
And a check that scans zero functions while the baseline holds entries is exit 2, not 0. A
scan path that does not exist was already an error; one that exists but matches nothing — a typo'd
paths, a src/→lib/ move, an over-broad exclude, or allow patterns that swallow everything —
used to report "No risk regressions detected" and pass forever. The message names which of the two
causes applies, because the fixes differ. Zero functions against an empty baseline is a legitimately
empty project and only warns, so a monorepo sweep over a package with no source yet still passes.
riskratchet baseline likewise refuses to overwrite a populated baseline with a zero-function scan
rather than discarding the ratchet; writing a fresh empty baseline still works.
A coverage file you named that does not exist is exit 2 as well. --coverage (and
--ts-coverage) is an assertion about this run, so a missing file cannot be right: riskratchet used
to fall through to auto-coverage, gate against a file it generated itself, and say nothing — which
turned a one-character typo into "No risk regressions detected", and made riskratchet baseline
anchor the ratchet to coverage nobody asked for. doctor already reported that setup as FAIL, so
the two commands disagreed about whether the same project was usable. A path from
[tool.riskratchet] coverage is different — it is a default auto-coverage may legitimately fill on a
fresh clone — so that one warns and names the file it used instead, and doctor reports it as
WARN for the same reason. --allow-missing-coverage downgrades the flag form to the same warning.
Finally, a report riskratchet cannot write — --output, --debug-json-file, or
baseline --output pointing at a directory or a read-only location — is exit 2, not the exit 1
plus traceback it used to be. Exit 1 means a gate tripped, and a full disk is not a gate.
For the same reason, a test command auto-coverage cannot start is exit 2. The default
test_command shells out to pytest; on a machine without it — a slim CI image, a project on a
different runner — that raised an uncaught FileNotFoundError and exited 1, so a missing test
runner was indistinguishable from a risk regression. An unparseable or empty test_command reports
the same way. A command that does run and then fails is unchanged: whatever coverage it wrote is
still used, because failing tests still produce a usable signal.
TypeScript
riskratchet scores TypeScript as a first-class backend alongside Python. Pass scan --typescript
(also check, diff, and baseline) and the functions in your .ts/.tsx/.mts/.cts files
are discovered, scored, gated, and tracked in the baseline exactly like Python — same score,
severity, CRAP, and components, flowing through the same functions[] array with
language: "typescript". It is opt-in (Python-only runs are untouched) and requires the
[typescript] extra.
Supported ahead of demand. Every TypeScript slice — discovery, complexity, coverage, public surface, token-stable fingerprints, and now scoring — was built by maintainer choice rather than in response to a paying-in-CI user. TypeScript is fully supported and technically complete, but its field-hardening is younger than the Python backend's; treat scores as calibrated-but-new. See
docs/language-backend-contract.mdfor what "supported" means per backend.
pip install 'riskratchet[typescript]' # opt-in extra (tree-sitter); Python-only installs are unaffected
riskratchet scan src --typescript
# TypeScript functions appear in the scan table (and functions[] / SARIF) tagged language=typescript.
--experimental-typescript is a deprecated hidden alias for --typescript, kept for one release; it
prints a deprecation warning and behaves identically. It will be removed in a later minor.
Turn it on from config or CI
Since 0.3.6 the switch lives in [tool.riskratchet] too, so every door — scan, check, diff,
baseline, explain, doctor, the pytest plugin, and the GitHub Action — reads the same
setting. ts_coverage and ts_entry are the config forms of --ts-coverage / --ts-entry,
relative to the config file; --no-typescript turns a configured true back off for one run.
riskratchet init writes typescript = true for you when it finds TypeScript under src.
[tool.riskratchet]
paths = ["src"]
typescript = true
ts_coverage = ["coverage/lcov.info"] # exit 2 on baseline/check/diff if missing; a warning on scan
ts_entry = ["src/index.ts"] # optional: narrow the public surface to one barrel
A TypeScript-only project needs no Python coverage: when TypeScript is on and there is no .py
file under the scan paths, riskratchet says so once and neither runs pytest nor asks for a
coverage.json. In CI, produce the report and hand it to the Action (which installs the
[typescript] extra itself):
# .github/workflows/riskratchet.yml — a TypeScript-only repo
on: [pull_request]
jobs:
riskratchet:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 0
- uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
node-version: 22
- run: npm ci
- run: npx vitest run --coverage --coverage.reporter=lcov # or c8 / nyc / jest
- uses: KayhanB21/riskratchet@v0.3.9
with:
typescript: 'true' # or leave empty and set `typescript = true` in config
ts-coverage: coverage/lcov.info
This workflow has no Python coverage step on purpose: with no .py file under the
scan paths, Python coverage is not applicable and riskratchet says so once instead of
failing. A mixed repo is a different case — Python coverage becomes applicable again,
so add setup-python, the install, and the pytest --cov step from the canonical
workflow above, and pass coverage: coverage.json alongside ts-coverage. Without them
a mixed repo exits 2, for the same reason the Python workflow needs its own step.
TypeScript complexity (the CC column) is the McCabe cyclomatic count, computed to match ESLint's
complexity rule: ?? counts as a branch but optional chaining ?. does not (it has no Python
counterpart and would inflate the count), switch default is not counted, and nested functions
are pruned so each is scored on its own. The raw count stays ESLint-faithful (so it matches a TS
dev's linter), while the structural_complexity score component normalizes it with a
corpus-derived TS calibration (docs/typescript-complexity-calibration.md),
so equal complexity percentiles score equally across Python and TypeScript.
Add --ts-coverage to annotate each function with line/branch coverage from an
Istanbul/nyc coverage-final.json (what nyc, c8, or Jest --coverage write) or an
LCOV lcov.info (what c8 --reporter=lcov, Karma, many Jest reporters, and CI uploaders
write). The format is auto-detected per file, so a single repeatable --ts-coverage list
may mix both — pass one report per package in a monorepo and they merge. It is separate from
Python --coverage. A file absent from the report is reported explicitly (not silently
dropped). If a report's line numbers don't line up with the source — the sign of coverage
collected on compiled JS without source-map remapping — riskratchet warns and omits that
file's coverage rather than showing wrong numbers. (TS line-coverage is statement-derived for
Istanbul and line-derived for LCOV, and neither is directly comparable to the Python line-level
percentage.)
Produce a report your toolchain already knows how to emit: c8 --reporter=lcov (LCOV) or
c8 --reporter=json / nyc --reporter=json (Istanbul); Jest via
coverageReporters: ['lcov'] (or ['json']); Vitest via --coverage.reporter=lcov; or
Karma's lcov reporter. Point --ts-coverage at the resulting lcov.info or
coverage-final.json — either works.
riskratchet scan src --typescript --ts-coverage coverage/coverage-final.json
# or an LCOV report — same output:
riskratchet scan src --typescript --ts-coverage coverage/lcov.info
Coverage feeds the coverage_gap / branch_gap score components and the CRAP number just as Python
coverage does. As noted above, each backend scores its own coverage fraction, so a TS percentage is
not directly comparable to the Python line-level percentage (documented, accepted).
Public surface is barrel-aware. By default is_public follows a file's own exports, but
when your package has an entry barrel riskratchet narrows it to what is actually reachable from
that entry through export … from re-exports (export { x } from, export * from,
transitively). The entry is taken from --ts-entry (repeatable), else package.json
(exports/module/main/types — best-effort, only when they point at source .ts; a
built-to-dist package falls through), else the shallowest index.{ts,tsx,mts,cts}. The entry
that drove narrowing is printed on stderr so you can see (and --ts-entry-override) the guess. A
file-exported function that no barrel re-exports is shown as [internal]. This only ever
narrows, never on an unproven graph: an unresolved export * (or missing entry) keeps all
flags, while a single unresolved named re-export (e.g. export { X } from 'some-lib') holds
just that name public and still narrows the rest — so a third-party re-export in your barrel
doesn't silently switch the whole feature off. Declaration merging and tsconfig alias resolution
are out of scope (they need the type checker).
riskratchet scan src --typescript --ts-entry src/index.ts
A --ts-entry that is not on disk is a setup error (exit 2) since 0.3.7 — a path you
named must exist. Before that it only warned that the entry "matched no scanned file"
and the run scored on with every file-level export flag intact, so a typo widened the
public surface silently. Command-line paths are relative to the current directory, not
to the config file. A [tool.riskratchet] ts_entry key keeps the warning instead: it is
a project default a fresh clone may not have generated yet, and failing on it would turn
a green gate red on a patch upgrade.
package.json entry fields are matched only when the value is written as a relative
specifier ("./index.ts"). A bare "index.ts" does not resolve today and falls through
to the index.{ts,tsx,mts,cts} heuristic — or, with no such file, to no narrowing at
all. Fixing that lowers public_surface wherever a barrel stops reaching a function, so
it waits for 0.4.0 with the other score-moving changes.
Machine-readable output. With --json, scored TypeScript functions ride in the same top-level
functions[] array as Python, each tagged language: "typescript" and carrying the identical
score/severity/CRAP/components shape (the key is absent for Python-only functions, which read
language: "python"):
// riskratchet scan src --typescript --ts-coverage coverage/lcov.info --json
{
"functions": [
{ "language": "python", /* … scored Python, same shape … */ },
{
"path": "src/math.ts", "qualname": "parseConfig", "language": "typescript",
"severity": "medium", "score": 43.75, "crap": 20.0, "complexity": 4,
"line_coverage": 0.0, "branch_coverage": null, "is_public": true,
"lines": {"start": 1, "end": 6},
"components": { "coverage_gap": 100.0, "structural_complexity": 15.0, "branch_gap": 0.0,
"churn": 0.0, "public_surface": 100.0, "sprawl": 0.0 }
}
]
}
Breaking (0.3.0): the separate informational top-level
typescript[]array that0.2.xemitted under--experimental-typescriptis gone. Scored TypeScript now flows throughfunctions[], sofunctions[].languageis"python"or"typescript"and a consumer that assumed every entry was Python must filter onlanguage.
Internally each function also carries a fingerprint (body) and signature — stable across
formatter whitespace/quote/paren choices, changing on real edits — that the baseline uses for
rename-aware tracking (see "Baseline format" below; TS identity is pinned to the tree-sitter grammar
version). With --format sarif, each scored TypeScript function is emitted under the shared
riskratchet.function-risk rule at a level mapped from its severity, tagged language: "typescript"
in properties — the same treatment Python functions get (the separate informational
riskratchet.typescript-function note rule is gone).
It discovers top-level functions, class methods (including on abstract and
anonymous default-export classes), and named (const/let-assigned) arrow and
function expressions; React function components fall out as exported
functions/arrows. Qualnames reflect nesting through classes, functions, and
namespace/module blocks, so a namespaced Foo.bar never collides with a
top-level bar. Public vs internal is export reachability — inline export /
export default, separate export { name } clauses, and (with an entry barrel) cross-file
re-export chains — not naming. Files with
syntax errors are skipped with a warning (never partially listed); if the skipped file
is the entry barrel itself, narrowing refuses ("the surface can't be bounded") and every
file-level export flag is kept rather than demoted. Deliberately
skipped: anonymous inline callbacks (xs.map(x => …)), object-literal methods,
interface/abstract method signatures (no body), and generated files (a
comment-anchored @generated header or *.pb.ts / *.gen.ts name) — counted in the
summary and listed in files[] with zero functions; a generated file still contributes
its exports, so a barrel's export * from './generated' keeps resolving. node_modules
is never descended into. Not yet
supported (silently skipped): generator functions and async iterators. The parser
is tree-sitter; the rationale and the contract a future backend must fill live in
docs/typescript-parser-decision.md and
docs/language-backend-contract.md.
Baseline format
The baseline (.riskratchet.json, format v3 since 0.3.0) is the ratchet's memory: one entry
per function, recording the score and components at the time it was written. check and diff
compare today's report against it. It is a reviewed artifact — commit it, and treat a bump the way
you'd treat a snapshot update. Its shape is validated by
schemas/baseline.schema.json.
What produced these numbers. Since 0.3.7 every baseline carries a top-level scoring
block recording the scoring-model version, the resolved weights, the churn window, whether
churn could be collected at all, and whether coverage was present. Two scores are only
comparable when the same five produced both, and a gate that compares them anyway is
reporting a difference nobody made. When they differ, check, diff, the pytest plugin,
and doctor's scoring-model row all say which one changed — and then compare anyway.
It is always a warning, never a failure: a mismatch makes the comparison untrustworthy,
not the code worse, and failing would break the upgrade the disclosure exists to protect.
Name the difference before you act on it. New weights in pyproject.toml mean "re-baseline,
that was deliberate". Churn that was collectable when the baseline was written and is not now
means the opposite — fix the repository access, because re-baselining bakes the zeroes in.
The same reasoning covers a baseline whose every entry has zero churn while the run measures
some; riskratchet calls that out separately, because it is the one case a pre-0.3.7 baseline
still betrays without any recorded provenance.
Baselines written by 0.3.0 through 0.3.6 carry no block and stay silent: format v3 already
implies the current scoring model, so there is nothing to report and nothing to nag about.
A v1 or v2 baseline — or one so old it has no version key — does warn, because those
predate 0.3.0's redefinition of sprawl and were genuinely scored by a different model.
Each entry carries path, qualname, score, components, plus a fingerprint (body) and
signature. Those two are stable across formatter whitespace/quote/paren choices and change on
real edits, which is what lets a renamed or moved function be tracked as the same function
instead of reported as one removal plus one new high-risk arrival.
Rename matcher: known limits. A body-fingerprint match plus any one other signal
(signature, path, qualname tail, component vector, score) is the bar for an unambiguous
match; signature-only matches are deliberately rejected, so a body rewrite cannot hide
behind a rename. The consequence: a function that is renamed and edited in the same
change cannot be matched, and is gated as new against fail_new_above, not as a
regression against its old score. Since 0.3.6 check says so when entries left the
baseline and others appeared (note: N functions left the baseline and M appeared …), and
riskratchet diff lists both sets. The weights and the 0.65 threshold are provisional —
expect the occasional ambiguous rename that needs the PR diff to resolve.
What check tells you about the baseline (since 0.3.6). Every format carries
Baseline: N entries · M compared · K not seen this run ("baseline": {entries, compared, removed} in --json), so a run that compared one entry of four can never read as a clean
bill of health. When entries it did not see live in files that still exist under the scanned
paths — an include / exclude hiding a baselined file, as opposed to a deleted file or a
deliberate riskratchet check packages/api subset — it warns on stderr with counts only
(safe under redaction) and names the two fixes. The verdict is never changed by either.
A baselined function that is still in the code (since 0.3.9). Four things take a function
out of the gate while its file stays in files[]: an allow pattern, a @generated header,
a file that fails to parse, and missing_coverage = "skip" for a file absent from the
coverage report. Before 0.3.9, check reported each one as removed function from baseline
and said nothing else. Now the Baseline: line counts them by cause, for example
3 not seen this run (1 suppressed by allow, 1 in a @generated file, 1 failed to parse).
check, diff, and the pytest plugin print one warning with the same counts. The diff
entry's reason names the cause, and its status stays removed, so the JSON schemas
don't change. The warning and the reason carry counts and causes only, never a path or a
pattern, so both are safe under redaction. The verdict doesn't change: as with exclude,
riskratchet can't tell a filter from an intent. A function you deleted keeps the old reason.
The identity block (v3). When a baseline contains TypeScript entries it also records
identity.typescript — the tree-sitter-typescript grammar version and the fingerprint scheme
version. Fingerprints hash grammar node-type strings, so a grammar upgrade legitimately changes
them. On a mismatch riskratchet does not treat that as a mass rename: it falls back to id-only
matching for TypeScript entries, warns, and (since 0.3.1) prints the exact regeneration command.
riskratchet doctor reports the same mismatch proactively. A Python-only baseline carries no
identity block and is byte-identical to v2, so upgrading changes nothing for Python-only users.
Version compatibility. This build reads baseline formats v1, v2, and v3, and writes v3, so
upgrading riskratchet never forces a re-baseline. Three cases are hard errors (exit 2) rather than
warnings, because each would otherwise leave the gate silently disengaged:
| Baseline | Result |
|---|---|
| A version newer than this build reads (e.g. v4) | Exit 2, telling you to upgrade riskratchet — a future entry shape would parse as zero entries and look clean. |
Not a JSON object, or no entries array |
Exit 2 — an unreadable baseline ratchets nothing. |
| Valid file, some individual entries unreadable | Runs, with a warning naming the count. Those functions are not ratcheted until you regenerate. |
Regenerate with riskratchet baseline (add --typescript if you scan TypeScript). If churn is
part of your scoring, regenerate in the same environment your gate runs in — churn depends on git
history, so a shallow clone or a different history produces different numbers.
Sample output on real libraries
I ran riskratchet against four widely-used Python libraries to show what its
output looks like on production code. Each was cloned fresh, its own test
suite run with pytest --cov --cov-report=json:coverage.json, then scanned.
Top findings:
| Library | Function | Score | CC | Line cov |
|---|---|---|---|---|
| python-slugify | __main__::main |
53.1 (high) | 3 | 11% (0% branch) |
| python-slugify | slugify |
33.3 | 27 | 88% |
| tabulate | _CustomTextWrap._wrap_chunks |
44.4 | 31 | 60% |
| tabulate | _normalize_tabular_data |
42.6 | 76 | 78% |
| tabulate | tabulate (entry) |
37.1 | 62 | 97% |
| humanize | precisedelta |
32.9 | 26 | 100% |
| humanize | naturaldelta |
32.4 | 33 | 100% |
| inflect | engine._sinoun |
36.7 | 108 | 98% |
| inflect | engine._plnoun |
36.2 | 100 | 99% |
The point is not that these libraries are bad. They have all-green CI and many users. The point is that even mature, well-tested code accumulates functions where complexity, coverage, and sprawl combine into something worth a second pair of eyes. A CC=108 function with 98% coverage is not on fire; it is a function that works and is tested. The ratchet's job is to keep those numbers from getting worse over time.
Comparison with other tools
| Tool | Per-function risk | Baseline / ratchet | Combines complexity + coverage + churn |
|---|---|---|---|
| coverage.py | line / branch only | no | no |
| radon | complexity only | no | no |
| xenon | complexity only | yes (threshold) | no |
| pytest-crap | yes (CRAP) | no | partial (CC + line coverage) |
| riskratchet | yes | yes | yes |
Local development
The same commands run in GitHub Actions:
uv sync --locked
uv run ruff check .
uv run ruff format --check .
uv run mypy src tests
uv run pytest --cov=src/riskratchet --cov-branch --cov-report=term-missing
uv build --clear
Strict typing covers both src/ and tests/.
Release files for riskratchet 0.3.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| riskratchet-0.3.9.tar.gz | 278.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| riskratchet-0.3.9-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 507.8 kB
Release files / riskratchet-0.3.9.tar.gz
| Download URL | riskratchet-0.3.9.tar.gz |
|---|---|
| Size | 278.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4b7c88692bb0ea25684db1a726126cb9ce66c2f002dc3cc5e3b7c2e16cf3d11b
|
|
BLAKE2b-256 checksum How to use checksums |
0ddd444e220d4f202419e847bda3944880fdcb82404638357eef7874e7002c41
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 26, 2026.
Transparency logRelease files / riskratchet-0.3.9-py3-none-any.whl
| Download URL | riskratchet-0.3.9-py3-none-any.whl |
|---|---|
| Size | 229.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b0659044a7cf9d142721bd2e5a6576cca9677fdd44df461c7cb2c81f2264c4b4
|
|
BLAKE2b-256 checksum How to use checksums |
9772b4ce8b19ecd87ca6c07dcd7431675a68d57fa676857233b31ceac3ece681
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 26, 2026.
Transparency log