CBP — Release-evidence gate

Prove the green build was not faked.

A reusable, self-hosted quality gate for repositories that do not want to trust a green check without evidence.

CBP is intentionally small: Python standard library at runtime, JSON/TOML configuration, and one CLI used by both GitHub Actions and local CI.

CBP is designed to evolve. As the CB ecosystem grows, CBP's rule engine will expand to cover different languages and frameworks.

Smooth upgrade is a promise. If you ship a breaking change without proper migration, CBP will break your pipeline.

CBP is designed for long-term stability. Upgrading CBP will not break your existing CI. New gates are opt-in by default, and legacy fields are auto-mapped.

The core 80% is language-neutral: encoding, documentation locks, release claims, dependency policy, and provenance. The remaining 20% is supplied by configured custom rules and evidence readers, so a React, Node.js, or future language project can extend the gate without changing CBP source.

What it catches

It is designed to make fake tests, fake coverage, dependency substitution, copy-paste comments, and edited evidence fail closed.

Coverage Gate ensures code paths are executed, but it cannot guarantee that assertions are semantically meaningful. CBP cannot validate the underlying execution of coverage.py; it validates the artifact and its freshness. If you need stronger provenance, enable require-coverage-signature with a coverage.json.sha256 sidecar generated by the same CI run.

GitHub Action

- uses: JunxiangPan/cbp-action@v1.0.3
  with:
    root: .
    config: .cbp/config.json
    coverage-json: coverage.json
    coverage-allowlist: .cbp/coverage_allowlist.json
    coverage-threshold: "80"
    policy-coverage-threshold: "85"
    dependency-lock: requirements.lock
    install-command: installer/install.ps1
    docs-manifest: .cbp/docs-manifest.json
    binary-artifacts: artifacts
    claims-manifest: quality-manifest.json
    claims-notes: RELEASE_NOTES.md
    enable-coverage: "true"
    enable-dependency: "true"
    enable-comments: "true"
    enable-docs: "true"
    enable-binary: "true"
    enable-claims: "false"
    mode: enforce

The output is written to $GITHUB_OUTPUT as:

result=CBP_RESULT=PASS

or CBP_RESULT=FAIL.

Inputs

Required path inputs

All path inputs are optional. A path is required only when its gate is enabled; the gate fails closed with an explicit missing-path error if it is enabled without the corresponding input.

Quick start

The canonical, copyable workflow is usage.yml. It runs coverage and then calls the Action with only the gates it enables.

Local CLI

python -m pip install -e ".[test]"
python -m cbp.cli --root /path/to/repo --config .cbp/config.json

audit mode reports failures but exits zero; enforce exits non-zero.

python -m cbp.cli --root . --mode audit
python -m cbp.cli --root . --mode enforce

Configuration

See schemas/config.schema.json.

{
  "coverage_json": "coverage.json",
  "coverage_allowlist": ".cbp/coverage_allowlist.json",
  "target_files": ["src/example.py"],
  "policy_file": "src/example.py",
  "coverage_threshold": 80,
  "policy_threshold": 85,
  "comment_files": ["src/example.py"],
  "enable_claims": false
}

All paths are relative to root. Every gate can be disabled with its enable_* field or the matching Action input.

Coverage groups

Coverage is configured as named groups. Each group can be enforce (a failure fails the Action) or warn (reported but non-blocking). This supports gradual adoption without a temporary lower global threshold.

{
  "coverage_json": "coverage.json",
  "coverage_allowlist": "coverage_allowlist.json",
  "coverage_groups": [
    {"name": "core", "files": ["src/core.py"], "threshold": 80, "mode": "enforce"},
    {"name": "auditors", "files": ["src/audit_*.py"], "threshold": 0, "mode": "warn"},
    {"name": "webui", "files": ["webui/**/*.py"], "threshold": 0, "mode": "warn"}
  ]
}

Suggested migration:

1. Start every new group with mode "warn" and threshold 0.
2. Add tests and raise the group threshold.
3. When the group is stable, change mode to "enforce".

target_files, policy_file, coverage_threshold, and policy_threshold remain supported for single-group configurations.

Self-test

cd cbp-action
python -m pip install -e ".[test]"
python -m coverage run --branch --source=src/cbp,tools -m pytest -q tests
python -m coverage json -o coverage.json
python -m cbp.cli --root . --mode enforce

The package has no runtime dependencies outside the Python standard library. pytest and coverage are optional test dependencies.

CBP is available on PyPI:

pip install cbp-action

The top-level import package is cbp for the action runtime.

The manifest is checksum + run-id pinned. Release wheels also carry GitHub OIDC build provenance from actions/attest-build-provenance. The manifest itself is not a signature; the wheel attestation is the cryptographic provenance layer.

security_core coverage is the coverage of the gate modules, config, CLI, encoding gate, and rule auditor; it is not a separate binary-security metric.

Configuration compatibility

Every supported config should declare:

"schema_version": "1.0.3"

Old v0.1.0 configs that omit it are accepted and migrated in memory with a CBP_CONFIG_WARN message. Legacy fields are mapped automatically:

installer_core -> install_command
broker_files   -> target_files
measured_files -> target_files

A future schema version newer than the running CBP is rejected rather than silently misread.

Deprecation is a three-step process:

v1.0.3: documentation + DeprecationWarning, old behavior still works
v1.1.0: old behavior still works, but emits a mandatory warning
v2.0.0: legacy fields may be removed after the warning period

New gates and fields are optional. A config with no custom_rules, evidence_readers, release_body, or coverage_signature still runs.

Cross-language custom rules

Evidence readers are declared in cbp.json and selected by suffix or explicitly by a rule. A reader is only a text decoder plus suffix/limit metadata; CBP does not import or execute third-party code.

{
  "evidence_readers": {
    "javascript": {"suffixes": [".js", ".jsx", ".ts", ".tsx"], "encoding": "utf-8"},
    "python": {"suffixes": [".py"], "encoding": "utf-8"}
  },
  "custom_rules": [
    {
      "name": "no-dangerous-html",
      "pattern": "dangerouslySetInnerHTML",
      "files": ["webui/**/*.jsx"],
      "reader": "javascript",
      "message": "Do not inject raw HTML.",
      "mode": "warn"
    },
    {
      "name": "no-node-eval",
      "pattern": "\beval\s*\(",
      "files": ["webui/**/*.js"],
      "reader": "javascript",
      "message": "Do not use eval().",
      "mode": "warn"
    }
  ]
}

Rules are regex or literal, support ignore_case, multiline, and dotall, and default to warn so a new language cannot break another project's CI.

Verifying build provenance

The release wheel and the manifest evidence are covered by GitHub OIDC attestation. Verify them from a checkout of the release repository:

gh attestation verify cbp_action-1.0.3-py3-none-any.whl   --repo JunxiangPan/cbp-action

gh attestation verify quality-manifest.json   --repo JunxiangPan/cbp-action

gh attestation verify quality-manifest.json.sha256   --repo JunxiangPan/cbp-action

CBP validates code execution paths, but cannot prove the semantic validity of test assertions.

Baseline and source binding

Old repositories can adopt CBP without fixing historical findings immediately:

python -m cbp.cli --root . --generate-baseline cbp-baseline.json

The baseline suppresses only the exact recorded findings; a new finding is still a failure. Inline suppression is also supported:

# cbp: ignore=CBP-CMT-001

Coverage evidence now embeds source_files_hash inside coverage.json. Coverage Gate recomputes the current measured-source hash and fails with:

coverage.json is stale; source files have changed

A touch coverage.json no longer bypasses the check. Generate and stamp it with:

python -m coverage json -o coverage.json
python tools/stamp_coverage.py --root . --coverage coverage.json

Release body and errata

docs/RELEASE_BODY.md is the only accepted source for the GitHub Release body. The release workflow publishes it with --notes-file and then runs:

python -m cbp.cli --verify-release v1.0.3 --repository owner/repo

The command fetches the live GitHub release body and compares its normalized SHA256 with the checked-in file. If the body was edited in the GitHub web UI, it exits non-zero with:

ERRATA release body mismatch ...

CBP cannot prevent an owner from editing a release on GitHub. But if you do, the evidence chain breaks. Always update docs/RELEASE_BODY.md, re-run CI, and publish a new tag (or a patch release). If you must edit an old release, the edit is only acceptable after the evidence is regenerated and checked in; a silent web edit is an ERRATA, not a valid update.

This is the open-source release discipline: edits are allowed, silent edits are not.

Protect tags

In GitHub, open Settings -> Tags -> New rule and protect released tags:

Tag pattern: v*
Allow deletions: off
Allow force pushes: off

Tag protection prevents accidental re-pointing of a released tag. It does not prevent editing a Release body; verify-release covers that gap.

Releasing

See RELEASING.md for the commit -> CI -> tag -> release order.

GitHub Code Scanning

- name: CBP SARIF
  run: python -m cbp.cli --root . --sarif-output cbp.sarif
- uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: cbp.sarif

Status

v1.0.3 is the current stable release. It does not import anything from the host product's quality/, checks/, or tools/ directories.

Coverage numbers in CI

CI coverage may differ from local by ±0.5% due to platform-specific line coverage. The gate uses the CI value.

PyPI

See docs/PYPI_TRUSTED_PUBLISHING.md; publishing uses GitHub OIDC and does not store a PyPI token.

Benchmark platform

The 6.44s 5k-file figure is Linux CI. On a Windows physical machine the same 5k run is about 30s because of filesystem stat overhead; this is documented and not treated as a Linux regression.

Prebuilt wheel

The action installs the checked-in dist/cbp_action-1.0.3-py3-none-any.whl; it does not build from source or contact the package index.

VM-only evidence

VM-only chaos evidence is planned for the v1.1.0 roadmap. A green CI run is not treated as a substitute for real VM acceptance; until the harness report is attached, this remains an explicit roadmap item.