verifiable-gates

Thai: README.th.md.

A registry of CI gates for projects built with or without AI coding agents. Every gate carries evidence of having gone red on a real defect, and the same rules ship as an agent skill, so the agent works under the constraints CI will later enforce. Of the 92 rules, nine have a checker in this package; the other 83 are written for an agent to read.

What this is not. Not a linter and not a SAST replacement. It decides configuration and process posture — pinning, CSP, ADR bookkeeping, supply chain — not code correctness. A green run means the nine checks it can decide found nothing; it says nothing about the 83 it cannot.

Quickstart

pip install verifiable-gates
cd your-project
python -m verifiable_gates.install .     # writes tools/, scaffold.json, gates.yaml, a starting workflow
python3 tools/gates_doctor.py            # runs the checkers the project now holds

What the two commands printed, in an empty git repository (2026-09-05, v0.3.0; the install line's absolute path is shortened to <your-project>):

$ python -m verifiable_gates.install .
installed into <your-project> — 9 gates (9 scan) · check with: python3 tools/gates_doctor.py
for the instruction file your agents read (AGENTS.md, CLAUDE.md), add one line: `run python3 tools/gates_doctor.py --rules before editing`
this bundle also carries the working: 10 practices, each with the lesson behind it and the pull requests it held on — off here; read `python3 tools/gates_doctor.py --working`, turn on with `install <dest> --working`
$ python3 tools/gates_doctor.py
[   NA] actions-sha-pinned — only the bundle's own starting workflow, untouched — nothing of yours to read
[   NA] adr-index-complete — no docs/adr — this rule reads the .md records and the README.md index under docs/adr (scaffold.json adr_path)
[   NA] ci-tools-hash-pinned — only the bundle's own starting workflow, untouched — nothing of yours to read
[   NA] csp-no-inline — no app/templates — this rule reads .html, .htm, .jinja, .jinja2 and .j2 templates under app/templates (scaffold.json templates_path)
[   NA] delete-means-soft-delete — no app — this rule reads Python modules under app (scaffold.json src_path) — session.delete calls outside the purge_paths
[ pass] gates-registry-total
[   NA] image-digest-pinned — no Dockerfile — this rule reads the FROM lines of the root Dockerfile (scaffold.json dockerfiles), and .github/dependabot.yml for a docker ecosystem
[   NA] logic-knows-no-http — no app/services — this rule reads Python modules under app/services (scaffold.json services_path) — their imports, for request-side symbols
[   NA] no-debug-entrypoint — no entrypoint — this rule reads the Python entrypoints run.py, wsgi.py, app.py and main.py (scaffold.json entrypoints), as an AST

waiting on this project's own tests: 0 gates
[exit 0]

On a fresh install the only pass is the registry index, which has to be true about itself. Everything else is NA until the project has something to check. Exit 0 here means nothing was measured, not that the project passed.

Add one workflow with a floating tag and an unpinned install, and the same command answers with two findings and exits 1 (the full transcript, with the third finding the new job itself causes, is in docs/output-semantics.md):

[found] actions-sha-pinned — Every action is pinned to a commit SHA with the version in a comment
  born from: Tags move, commits do not — upload-artifact once sat on @v4 at a single call site for ten days with CI green throughout.
actions-sha-pinned: .github/workflows/lint.yml: actions/checkout@v4
…
[found] ci-tools-hash-pinned — Tools CI installs for itself are pinned by hash, on both the Python and the Node side
  born from: An unpinned install command takes whatever is newest at the second the job runs, and it runs with our workflow's privileges · pinning one package at a time pins only that package while the rest of the tree still floats.
ci-tools-hash-pinned: .github/workflows/lint.yml: pip install ruff
…
** scans found problems in 3 gates: actions-sha-pinned, ci-tools-hash-pinned, gates-registry-total
[exit 1]

Reading the output

Two consequences to know before trusting a green:

• A project where every rule is NA exits 0. That is an unmeasured project, not a passed one (DECISIONS.md doctor-all-na-exits-zero).
• A path that scaffold.json names and the project does not have is a finding, not NA: a broken configuration is a defect, not an absence.
• Saying "not yet" goes through the record, not around it. There is no # noqa here and no warn-only mode; a finding a project will not fix this week gets a waiver in scaffold.json (waivers: [{gate, reason, until, decided_by, scope?}]), which the doctor prints on every run, turns red when it expires, and refuses when a field is missing. The SARIF keeps the result, with the reason as its suppression.

The full taxonomy — every NA and [error] case, the --installed record, --rules off an edited bundle, the SARIF mapping — is in docs/output-semantics.md.

Three ways to run it

What it costs. The doctor is about 0.4 s of interpreter starts — nine scans, nine processes — plus roughly 1.6 ms per Python file under src_path: 0.5 s on this repository (98 modules), 2.5 s at 907, and 14.5 s at 7 256 (measured 2026-09-05, one laptop; the numbers and the method are in .local/ of that day's audit round). Most of it is one scan, delete-means-soft-delete, which reads every module. Every pre-commit hook here is always_run, so each commit re-reads the whole tree however small the change — on a large tree prefer the action, or the one hook whose rule you care about. There is no timing gate: a threshold on wall clock is a ratchet against the runner's hardware, not against the code (DECISIONS.md the-cost-is-stated-and-not-gated).

All three run tools/ as the project has it, and none carries a copy of the checkers. Moving the SHA, the rev or the plugin version changes nothing about what the project is held to (DECISIONS.md ci-runs-the-bundle-the-project-installed).

The action is listed on the GitHub Marketplace as verifiable-gates — pin the SHA, not the tag the listing offers. The edit hook hands a finding back to the agent while it still holds the file, and refuses nothing (DECISIONS.md the-edit-hook-reports-and-does-not-refuse).

What the nine checkers decide

Each of the nine stdlib-only checkers is one Python file under tools/checks/, run by the doctor or on its own; nothing the bundle installs reaches off this machine (its one socket asks whether a service is listening on 127.0.0.1). python3 tools/gates_doctor.py --rules prints this list off the installed bundle, with each rule's incident.

The paths are the defaults scaffold.json carries; the project moves them there.

The 92 rules

rules.yaml — 92 rules, each carrying the incident that produced it (born_from), because a rule with no origin is a rule nobody knows when to remove. They are rendered into an agent skill in the layout of the Agent Skills specification: skills/verifiable-gates/SKILL.md is the front page, and the full entries sit beside it in references/.

Two pipes this repository does not own install the skill without cloning:

The nine rules with a checker (script: in rules.yaml) are the ones the doctor and the installer decide, and nothing else. The other 83 are the rule sheets an agent is held to by reading, and the Enforced in the reference line on each says how one project turned it into a test. What each pipe sends and fetches, and why there is no registry of this project's own, is in docs/history.md and DECISIONS.md distribution-is-two-pipes-nobody-here-owns.

A rule that turns out to be wrong is withdrawn in place, never deleted: retracted: keeps it in the catalogue and on its sheet with the date and the reason, marked and out of every count, so a reader who followed it can find out that they should stop. Nothing is withdrawn today.

A rule may also say where it sits in a vocabulary somebody else already speaks: maps_to: names items of OpenSSF Scorecard, SLSA v1.0 and NIST SSDF — 35 of them name where they sit, and the rest deliberately name nothing, because no item of those three covers them. A mapping says the rule would satisfy or contribute to that item, never that the two are equal; the item names are a closed set read off the publications, so a misspelling is refused rather than published as a map to nothing.

rules.yaml and the sheets come with the checkout, not with the wheel. The package is the machinery that reads the catalogue:

import verifiable_gates

catalogue = verifiable_gates.rules.load("rules.yaml")
# `package_dir` is where a rule's `script:` is looked for; without it, only the
# shape of the path is checked — a checker that is not there would go unnoticed.
for problem in verifiable_gates.rules.problems(catalogue, package_dir="src/verifiable_gates"):
    print(problem)

Design constraints

The two schemas — rules.py for this catalogue, registry.py for a project's gates.yaml — encode five rules that came from real traps, not from theory:

• a gate whose layer is internal cannot be portable — a rule tied to one project's architecture, exported as universal, is an overclaim; that hold is in the gate schema, and the rule schema refuses internal outright, since a rule in this catalogue is published whole (portable on a rule is refused as a gate's field);
• a key neither schema knows is refused, not skipped — a misspelt born_frm is a rule with no origin that looks like one with;
• anything exported must name the trap that created it (born_from), because a rule with no origin is a rule nobody knows when to remove;
• proved_by entries must say what they caught and when — a gate nobody has seen go red is indistinguishable from a gate that checks nothing;
• the vocabularies for kind, severity, layer, and pillar are closed — and a severity is what something reads: blocking is every gate on a run (a scan is red whatever the row says), watched is a watched_by promise the red-streak census measures, and there is no third word, because a label no run reads is a softness nobody delivers.

A rule and its enforcement live in separate files, because they have separate lifetimes: rules.yaml is what this project publishes; gates.yaml is what this project is itself held to. What was deliberately not done, each with the condition that would expire it, is DECISIONS.md.

Provenance and status

Archived at doi:10.5281/zenodo.22103110, which resolves to the latest version; each release also gets a DOI of its own. The tag evidence-freeze-1 is the state the measurements were taken on and v0.1.0 (2026-08-28) the state the package first shipped in — different commits on purpose. The extraction from the reference implementation, sayam/flask-todolist, is complete; the stage table, the census and what stayed behind are in docs/history.md. Consume a pinned submodule or a versioned dependency, never main.

None of this has to be taken on trust. docs/auditing.md is the hour: the eleven rules this repository states about itself, the command that decides each one, and — said plainly — what no command here can answer.

Who maintains this

One maintainer, and no promise about how long. What survives if that stops — your CI, your alerts' ids, and a fork that is complete — is in GOVERNANCE.md, together with how a rule is withdrawn and how you would tell whether anybody is still here.

Licence

• Code: Apache-2.0. Contributors sign CLA.md — one line in the pull request; you keep your copyright.
• Rules and documentation: CC BY 4.0.

The application this was extracted from stays AGPL-3.0-or-later. The two differ on purpose: a CI tool is not a network service, and a rule meant to be adopted inside an organisation's internal handbook must not require share-alike.

README.th.md · docs/ · CONTRIBUTING.md · SECURITY.md · CHANGELOG.md · the experiment