CBP — Release-evidence gate

Prove the green build was not faked.

A reusable, self-hosted quality gate for repositories that do not want to trust a green check without evidence.

CBP keeps a small surface area with zero runtime dependencies: Python standard library only, JSON/TOML configuration, and one CLI used by both GitHub Actions and local CI.

CBP is designed to evolve. As the CB ecosystem grows, CBP's rule engine will expand to cover different languages and frameworks.

Smooth upgrade is a promise. If you ship a breaking change without proper migration, CBP will break your pipeline.

CBP is designed for long-term stability. Upgrading CBP will not break your existing CI. New gates are opt-in by default, and legacy fields are auto-mapped.

The core 80% is language-neutral: encoding, documentation locks, release claims, dependency policy, and provenance. The remaining 20% is supplied by configured custom rules and evidence readers, so a React, Node.js, or future language project can extend the gate without changing CBP source.

What it catches

It is designed to make fake tests, fake coverage, dependency substitution, copy-paste comments, and edited evidence fail closed.

Coverage Gate ensures code paths are executed, but it cannot guarantee that assertions are semantically meaningful. CBP cannot validate the underlying execution of coverage.py; it validates the artifact and its freshness. If you need stronger provenance, enable require-coverage-signature with a coverage.json.sha256 sidecar generated by the same CI run.

GitHub Action

- uses: JunxiangPan/cbp-action@v1.0.3
  with:
    root: .
    config: .cbp/config.json
    coverage-json: coverage.json
    coverage-allowlist: .cbp/coverage_allowlist.json
    coverage-threshold: "80"
    policy-coverage-threshold: "85"
    dependency-lock: requirements.lock
    install-command: installer/install.ps1
    docs-manifest: .cbp/docs-manifest.json
    binary-artifacts: artifacts
    claims-manifest: quality-manifest.json
    claims-notes: RELEASE_NOTES.md
    enable-coverage: "true"
    enable-dependency: "true"
    enable-comments: "true"
    enable-docs: "true"
    enable-binary: "true"
    enable-claims: "false"
    mode: enforce

The output is written to $GITHUB_OUTPUT as:

result=CBP_RESULT=PASS

or CBP_RESULT=FAIL.

Inputs

Required path inputs

All path inputs are optional. A path is required only when its gate is enabled; the gate fails closed with an explicit missing-path error if it is enabled without the corresponding input.

Quick start

The canonical, copyable workflow is usage.yml. It runs coverage and then calls the Action with only the gates it enables.

Local CLI

python -m pip install -e ".[test]"
python -m cbp.cli --root /path/to/repo --config .cbp/config.json

Exit codes follow CONTRACT v1.7:

audit and warn are local diagnostics: they exit 0 but print CBP_MODE=… contract_exit_code=… publishable=…, so a suppressed failure stays visible. Only enforce produces publishable evidence.

python -m cbp.cli --root . --mode audit
python -m cbp.cli --root . --mode enforce --report-json gate-report.json
python -m cbp.cli --parse-report gate-report.json
python -m cbp.cli --parse-report other-report.json --observed-exit-code 3

--parse-report reads structured JSON only; it never greps stdout or a log. An exit code of 3 without a valid gate_status is treated as an input error (exit 2), per CONTRACT §1.

When package_type is cbp, the gates claims, binary, encoding, docs, and custom are forced: a SKIPPED forced gate is a blocking skip and exits 3. skip_ratio = skip_count / applicable_gate_count; more than 20% exits 4.

Historical release bodies can be corrected without hiding the change. F4 reads only a body that starts with ERRATUM:; a section appended at the end is human-readable but invisible to the machine rule.

python -m cbp.cli errata --body current.md --output corrected.md --reason "..."
python -m cbp.cli errata --body corrected.md --check

cbp errata generates and validates the body, including byte-preserving the original text. It never edits GitHub: the owner reviews the diff and runs gh release edit <tag> --notes-file corrected.md.

Rule packs

python -m cbp.cli --root . --rules-path rules/cbe-pack.json --rules-sha256 <hex>

The pack is loaded into the custom gate. A missing file, invalid JSON, a missing name/version/rules, or a hash mismatch is an input error (exit 2). The action exposes the same knobs as rules-path / rules-sha256.

Encoding allowlist

Windows PowerShell 5.1 needs a BOM to read non-ASCII scripts. Such files can be exempted one path at a time, with a reason; there are no wildcards.

"encoding_allowlist": [
  {"path": "build/legacy.ps1", "reason": "Windows PowerShell 5.1 requires BOM for CJK"}
]

Absolute paths, wildcards, duplicate entries, missing reasons, and entries that match no tracked text file are rejected. A passing run reports allowlisted=N; an allowlisted file without a BOM is reported as allowlist_unused=N.

Release asset manifest

--verify-release reads <root>/dist/RELEASE_ASSETS.sha256 by default. Pass --release-assets PATH when the inventory lives elsewhere; a missing file is an input error (exit 2). Network failures (timeouts, HTTP 5xx) are infrastructure failures: exit 3 with GATE_STATUS=SKIPPED_INFRA, and publish stays blocked.

1.2.0 rules

python -m cbp.cli --root . --v12 --v12-report v12-report.json
python -m cbp.cli --root . --v12 --v12-only CBP-CI-001
python -m cbp.cli --root . --v12 --waivers waivers.json --v12-baseline baseline.json

Thirteen rules: seven blocking (CBP-CI-001, CBP-VER-001, CBP-QA-001, CBP-DEBT-001, CBP-SIGN-001, CBP-WT-001, CBP-BODY-001) and six warning (CBP-BIN-001, CBP-DOC-001, CBP-CLAIM-001, CBP-COV-001, CBP-PROOF-001, CBP-SLOP-001). Blocking failure exits 3; warning-only failure exits 0 with a visible WARN until 1.2.1.

A waiver or baseline entry needs all four fields; an expired one is not applied and is printed:

{"rule": "CBP-DOC-001", "path": "docs/old.md", "reason": "archived", "expires": "2026-12-31"}

body_hash_scope and artifact_repository

The manifest now carries body_hash_scope: "base" (only base is supported; full is exit 2) and, when configured, artifact_repository. Pass --artifact-repository owner/repo or set GITHUB_REPOSITORY; a consumer can reject a cross-repository artifact with --parse-report --expected-repository owner/repo.

cbc-event

Emission is disabled by default. Enable a local JSONL chain with:

python -m cbp.cli --root . --cbc-event-dir .cbp/events --cbc-event-max 500

Events are cbp.run.start, cbp.gate.result and cbp.run.end, chained by prev_hash / event_hash. --cbc-event-hmac-env NAME reads an HMAC-SHA256 key from the environment. Dropped events are counted and printed, never silent.

External-package manifest

cbp manifest writes a CONTRACT §5 manifest for a subject package (cbc, cbm, ...) using a real gate report plus that package's artifacts and body:

python -m cbp.cli manifest \
  --package-id cbc --producer-id cbp --version v0.1.0 \
  --gate-report gate-report.json \
  --artifact dist/pkg-0.1.0-py3-none-any.whl \
  --artifact dist/pkg-0.1.0.tar.gz \
  --release-body RELEASE_BODY.md \
  --body-hash-scope base \
  --out quality-manifest.json

--artifact and --evidence are repeatable; --artifact-sha256 NAME=HEX checks an expected digest. Missing gate report / artifact / body / evidence, an unsupported scope, or a digest mismatch is exit 2 and writes nothing. A failing gate report is preserved as FAIL, never rewritten to PASS. body_hash_scope: base hashes the bytes before <!-- CBC-EVIDENCE-BEGIN --> when present; otherwise the whole body.

Configuration

See schemas/config.schema.json.

{
  "coverage_json": "coverage.json",
  "coverage_allowlist": ".cbp/coverage_allowlist.json",
  "target_files": ["src/example.py"],
  "policy_file": "src/example.py",
  "coverage_threshold": 80,
  "policy_threshold": 85,
  "comment_files": ["src/example.py"],
  "enable_claims": false
}

All paths are relative to root. Every gate can be disabled with its enable_* field or the matching Action input.

Coverage groups

Coverage is configured as named groups. Each group can be enforce (a failure fails the Action) or warn (reported but non-blocking). This supports gradual adoption without a temporary lower global threshold.

{
  "coverage_json": "coverage.json",
  "coverage_allowlist": "coverage_allowlist.json",
  "coverage_groups": [
    {"name": "core", "files": ["src/core.py"], "threshold": 80, "mode": "enforce"},
    {"name": "auditors", "files": ["src/audit_*.py"], "threshold": 0, "mode": "warn"},
    {"name": "webui", "files": ["webui/**/*.py"], "threshold": 0, "mode": "warn"}
  ]
}

Suggested migration:

1. Start every new group with mode "warn" and threshold 0.
2. Add tests and raise the group threshold.
3. When the group is stable, change mode to "enforce".

target_files, policy_file, coverage_threshold, and policy_threshold remain supported for single-group configurations.

Self-test

cd cbp-action
python -m pip install -e ".[test]"
python -m build --wheel --sdist --outdir dist
python -m coverage run --branch --source=src/cbp,tools -m pytest -q tests
python -m coverage json -o coverage.json
python tools/stamp_coverage.py --root . --coverage coverage.json
python tools/dogfood_manifest.py --root=. --coverage-json=coverage.json --output=quality-manifest.json --notes-template=docs/RELEASE_NOTES.md --notes-output=docs/RELEASE_NOTES.generated.md
python -m cbp.cli --root . --mode enforce

The build and stamp steps are mandatory: the claims gate reads dist/RELEASE_BODY.md and the wheel it names, and the coverage gate rejects an unstamped coverage.json. The repository tracks neither file.

The package has no runtime dependencies outside the Python standard library. pytest and coverage are optional test dependencies.

CBP is available on PyPI:

pip install cbp-action

The top-level import package is cbp for the action runtime.

The manifest is checksum + run-id pinned. Release wheels also carry GitHub OIDC build provenance from actions/attest-build-provenance. The manifest itself is not a signature; the wheel attestation is the cryptographic provenance layer.

security_core coverage is the coverage of the gate modules, config, CLI, encoding gate, and rule auditor; it is not a separate binary-security metric.

Configuration compatibility

Every supported config should declare:

"schema_version": "1.0.3"

Old v0.1.0 configs that omit it are accepted and migrated in memory with a CBP_CONFIG_WARN message. Legacy fields are mapped automatically:

installer_core -> install_command
broker_files   -> target_files
measured_files -> target_files

A future schema version newer than the running CBP is rejected rather than silently misread.

Deprecation is a three-step process:

v1.0.3: documentation + DeprecationWarning, old behavior still works
v1.1.0: old behavior still works, but emits a mandatory warning
v2.0.0: legacy fields may be removed after the warning period

New gates and fields are optional. A config with no custom_rules, evidence_readers, release_body, or coverage_signature still runs.

Cross-language custom rules

Evidence readers are declared in cbp.json and selected by suffix or explicitly by a rule. A reader is only a text decoder plus suffix/limit metadata; CBP does not import or execute third-party code.

{
  "evidence_readers": {
    "javascript": {"suffixes": [".js", ".jsx", ".ts", ".tsx"], "encoding": "utf-8"},
    "python": {"suffixes": [".py"], "encoding": "utf-8"}
  },
  "custom_rules": [
    {
      "name": "no-dangerous-html",
      "pattern": "dangerouslySetInnerHTML",
      "files": ["webui/**/*.jsx"],
      "reader": "javascript",
      "message": "Do not inject raw HTML.",
      "mode": "warn"
    },
    {
      "name": "no-node-eval",
      "pattern": "\beval\s*\(",
      "files": ["webui/**/*.js"],
      "reader": "javascript",
      "message": "Do not use eval().",
      "mode": "warn"
    }
  ]
}

Rules are regex or literal, support ignore_case, multiline, and dotall, and default to warn so a new language cannot break another project's CI.

Verifying build provenance

The release wheel and the manifest evidence are covered by GitHub OIDC attestation. Verify them from a checkout of the release repository:

gh attestation verify cbp_action-1.0.3-py3-none-any.whl   --repo JunxiangPan/cbp-action

gh attestation verify quality-manifest.json   --repo JunxiangPan/cbp-action

gh attestation verify quality-manifest.json.sha256   --repo JunxiangPan/cbp-action

CBP validates code execution paths, but cannot prove the semantic validity of test assertions.

Baseline and source binding

Old repositories can adopt CBP without fixing historical findings immediately:

python -m cbp.cli --root . --generate-baseline cbp-baseline.json

The baseline suppresses only the exact recorded findings; a new finding is still a failure. Inline suppression is also supported:

# cbp: ignore=CBP-CMT-001

Coverage evidence now embeds source_files_hash inside coverage.json. Coverage Gate recomputes the current measured-source hash and fails with:

coverage.json is stale; source files have changed

A touch coverage.json no longer bypasses the check. Generate and stamp it with:

python -m coverage json -o coverage.json
python tools/stamp_coverage.py --root . --coverage coverage.json

Release body and errata

The release body is generated from the built wheel by tools/generate_release_body.py during the publish job. That run uploads the exact body it published as the release-body-<run_id> artifact, and clean-verify checks the live body against that artifact, not against a file in the tag. This keeps the check meaningful without moving a tag to match a body that could only be generated after the build.

docs/RELEASE_BODY.md mirrors the published body for readers and for local checks. --verify-release compares the live body with whatever local body you pass (default: docs/RELEASE_BODY.md) and exits non-zero on a difference:

python -m cbp.cli --verify-release v1.0.9 --repository owner/repo

ERRATA release body mismatch ...

CBP cannot prevent an owner from editing a release on GitHub. But if you do, the evidence chain breaks: record the edit as an ERRATUM in the body itself, checked in through docs/RELEASE_BODY.md, and do not silently patch it. A silent web edit is an ERRATA, not a valid update.

This is the open-source release discipline: edits are allowed, silent edits are not.

Protect tags

In GitHub, open Settings -> Tags -> New rule and protect released tags:

Tag pattern: v*
Allow deletions: off
Allow force pushes: off

Tag protection prevents accidental re-pointing of a released tag. It does not prevent editing a Release body; verify-release covers that gap.

Action pinning

Every uses: in a workflow must be pinned to a full 40-character commit SHA (for example actions/checkout@11d5960a326750d5838078e36cf38b85af677262). A tag or branch pin (@v4, @main) is a mutable reference and is rejected by the artifact-integrity check with exit 2 and the message GitHub Action is not pinned to a full SHA. Pin the commit and keep the tag in a comment if you want the readable version. usage.yml, docs/ and examples/ are documentation and are not scanned.

Quality / AI evidence gates (1.5.0, opt-in)

CBP 1.5.0 adds opt-in --target-mutation, --test-quality and --ai-evidence gates. They are independent of CBE, do not change the default 1.4.0 behavior, and never modify the frozen quality manifest. See docs/1.5.0/QUALITY_GATES.md.

Strict profile (1.4.0, opt-in)

--profile strict requires signed provenance (cosign/OIDC), the gate manifest, mutation/hostile/fuzz evidence, full-SHA action pins, lockfile integrity, tag map immutability, Windows CI, an SBOM and the external provenance verifier. It writes a strict-profile.json sidecar and never changes the quality-manifest schema or the default 1.3.2 behavior. See docs/1.4.0/STRICT_PROFILE.md.

Releasing

See RELEASING.md for the commit -> CI -> tag -> release order.

GitHub Code Scanning

- name: CBP SARIF
  run: python -m cbp.cli --root . --sarif-output cbp.sarif
- uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: cbp.sarif

Status

The current stable release is the latest entry on the GitHub Releases page. CBP does not import anything from the host product's quality/, checks/, or tools/ directories.

Coverage numbers in CI

CI coverage may differ from local by ±0.5% due to platform-specific line coverage. The gate uses the CI value.

PyPI

See docs/PYPI_TRUSTED_PUBLISHING.md; publishing uses GitHub OIDC and does not store a PyPI token.

Benchmark platform

The 6.44s 5k-file figure is Linux CI. On a Windows physical machine the same 5k run is about 30s because of filesystem stat overhead; this is documented and not treated as a Linux regression.

Action runtime wheel

The repository tracks no wheel, no sdist, and no WHEEL_SHA256. The composite action resolves its release tag (cbp-version, or the tag it was pinned to), downloads that release's wheel from GitHub, checks its SHA256 against the release asset digest reported by the GitHub API, verifies the Sigstore provenance with gh attestation verify, and only then installs it. The verified digest and path are exposed as the wheel-sha256 and wheel-path outputs so a caller can assert that the wheel it ran is the wheel that was published.

The action therefore needs network access to GitHub Releases and gh on the runner. It fails closed: no digest, no attestation, or a digest mismatch aborts the step instead of falling back to an unverified artifact. Pin the action to a release tag; a branch or local-path reference must pass cbp-version explicitly.

The build-artifact job in ci.yml is the only place that runs python -m build. Every other job downloads the cbp-dist artifact.

VM-only evidence

VM-only chaos evidence remains an explicit gap. v1.2.0 is backed by CI, RC attestation and consumer parity, not by a VM acceptance report; do not describe CI evidence as VM acceptance.