banza-conformance 0.1.0

BANZA protocol conformance runner — generates conformance evidence (not certification) for an operator endpoint.

Banza Conformance Runner

Tests a running Banza operator against the protocol conformance suite and produces a structured JSON report.

The runner generates conformance evidence, not certification. A PASS is technical conformance evidence for the requested level — it is not a production certificate. Production certification remains M2/M3-gated and is issued separately by the BANZA CA.

Distribution for external operators

External operators can run BANZA conformance against their public sandbox endpoint without cloning this repository, in three ways.

Installable CLI

pip install banza-conformance        # from tools/banza-conformance
banza-conformance --url https://sandbox.operator.example --level 0 --output report.json

Supported options: --url (required), --level, --suite, --output, --quiet, --federation (L3, requires the federation extra: pip install "banza-conformance[federation]"). The CLI calls the same conformance logic as run.py — python3 tools/banza-conformance/run.py … continues to work unchanged.

On completion the tool prints:

Conformance report generated. This is evidence, not certification.

Client identity (User-Agent)

The runner sends a stable, documented User-Agent so operators can allow it in WAF / proxy / CDN configurations:

BANZA-Conformance/1.0 (+https://banza.org/conformance)

If your sandbox endpoint sits behind a bot filter (e.g. Cloudflare), allow this User-Agent (or the /health and /.well-known/banza/* paths) so the runner can reach it. The User-Agent identifies the client cleanly; it does not bypass any security control.

Docker

docker run --rm \
  -v "$PWD/reports:/reports" \
  ghcr.io/banza-protocol/banza-conformance:latest \
  --url https://sandbox.operator.example \
  --level 0 \
  --output /reports/banza-l0-report.json

Build the image locally (context = this directory):

docker build -t banza-conformance:local tools/banza-conformance

The entrypoint is banza-conformance; arguments after the image name are passed straight through. Reports are written under /reports — mount a host volume there.

GitHub Action

Run BANZA conformance in your CI/CD against your public sandbox endpoint:

name: BANZA Conformance

on:
  workflow_dispatch:
  pull_request:
  push:

jobs:
  conformance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: banza-protocol/banza/.github/actions/banza-conformance@main
        with:
          url: https://sandbox.operator.example
          level: "0"
          output: banza-l0-report.json
      - uses: actions/upload-artifact@v4
        with:
          name: banza-conformance-report
          path: banza-l0-report.json

Inputs: url (required), level (default "0"), output (default "banza-conformance-report.json"), suite, federation, quiet.

Requirements

Python 3.8+. No third-party packages for the core runner — stdlib only.

For L3 federation conformance (signature verification): pip install cryptography>=41.0.0

Usage

Standard conformance (L0–L4)

# Run all suites against a local operator
python3 tools/banza-conformance/run.py --url http://localhost:3000

# Run up to Level 2 only
python3 tools/banza-conformance/run.py --url http://localhost:3000 --level 2

# Run a single suite
python3 tools/banza-conformance/run.py --url http://localhost:3000 --suite health

# Write report to file (quiet mode)
python3 tools/banza-conformance/run.py --url http://localhost:3000 --output report.json --quiet

Or via the shell wrapper:

./tools/banza-conformance/run.sh --url http://localhost:3000 --output report.json

L3 Federation Conformance (79 tests, 9 suites)

The federation conformance suite tests cross-operator federation protocol compliance. It requires two terminals: one for the fixture server (Operator A adapter) and one for the runner.

# Terminal 1 — start the fixture server (Operator A)
python3 tools/banza-conformance/fixture_server.py --port 8099

# Terminal 2 — run the full federation suite (79 tests)
python3 tools/banza-conformance/run.py \
  --federation \
  --url http://localhost:8099 \
  --output l3-report.json

# Run only one federation sub-suite
python3 tools/banza-conformance/run.py \
  --federation \
  --url http://localhost:8099 \
  --fed-suite cert        # cert | disc | trust | route | exec | obl | evt | settle | fail

What the federation runner does:

• Generates an ephemeral test BANZA root keypair (never reused as production)
• Spins up an embedded Simulated Operator B (random port, in-process)
• Spins up an embedded test BRL/key-manifest server (random port, in-process)
• Runs all 79 federation tests against the fixture server
• Produces a structured JSON evidence report

Federation suites:

Suite	Cases	Blocking	What it tests
FED-CERT	11	Yes	Certificate schema, signature, expiry, BRL
FED-DISC	8	Yes	Federation manifest extension fields
FED-TRUST	9	Yes	All 9 ADR-026 trust protocol steps
FED-ROUTE	12	Yes	Routing wire protocol, idempotency, rejections
FED-EXEC	8	Yes	Acceptance semantics, ledger atomicity
FED-OBL	7	Yes	Obligation creation, signature, state machine
FED-EVT	6	No	Federation event emission and schema
FED-SETTLE	10	No	Netting, settlement, reconciliation
FED-FAIL	8	No	Retry, crash recovery, revocation, mismatches

L3 certification decision: All blocking suites (FED-CERT through FED-OBL) must PASS. FAIL in a blocking suite denies certification. FAIL in a non-blocking suite produces conditional certification (30-day remediation window).

See docs/federation/FEDERATION_CONFORMANCE_MODEL.md for the full decision rules.

Options

Flag	Description
--url	Base URL of the operator (required)
--level	Maximum certification level to test, 0–4 (default 4)
--suite	Run one suite only: health, wallets, transfers, traces, manifest
--output	Write JSON report to this path (default: stdout)
--quiet	Suppress PASS lines; only print failures and summary

Certification levels

Level	Name	Runner suites	Awarded here
L0	Protocol Sandbox	health, manifest (sandbox-safety)	Yes
L1	Core Payment Capability	+ wallets, transfers, traces (traceability)	Yes
L2	Payment Initiation Capability	+ payment_initiation (payment requests, dynamic QR, instant execution)	Yes
L3	Inter-Operator Interoperability	federation suite — run via --federation (run_fed.py)	No (multi-operator)
L4	External Interoperability	External Interoperability profile (external-rail evidence)	No (profile)

Level names and per-level capabilities are canonical and match BANZA_CERTIFICATION.md § Conformance level model and ADR-034. This single-operator sandbox runner awards L0–L2; L3 (federation) and L4 (external) require multi-operator / external-rail evidence and are not awarded here.

Exit codes

Code	Meaning
0	All executed cases passed
1	One or more cases failed
2	Runner error (connectivity, bad arguments)

Report format

Reports conform to conformance/report-schema.json. Each case records:

• status: PASS / FAIL / SKIP / ERROR
• request and response for HTTP cases
• assertions: per-field check results
• failure_reason: human-readable explanation on failure

Adding new suites

1. Implement a runner function run_<name>(base_url, ctx) -> list[CaseResult]
2. Register it in the SUITES dict
3. Add it to the relevant LEVEL_SUITES entries
4. Add the corresponding vectors in conformance/vectors/