Witnessbox

Evidence-based testing (EBT) of application behaviour: during tests the code records which decision it made and on what data, and an independent checker re-verifies every decision against a human-approved specification. The kit consists of a run gate, a checker and SDKs for Java, Dart and TypeScript. The runtime requires Python 3.11+ and has no third-party dependencies. The gate is a bash script tested on macOS and Linux; native Windows is not supported, and running it under WSL is not covered by tests.

Quick start

uv tool install witnessbox          # or: pipx install witnessbox
ebt-gate --scaffold ts ebt          # ebt/ scaffold for the dart, ts or jvm stack
ebt-gate ebt /path/to/clean-checkout /path/to/results

The scaffold contains an example spec.toml, checks.py and a stack-specific test.sh. A human approves the specification before the first run, and test.sh wires in the project's tests (new project). The installation provides ebt-gate and ebt-check; there is no separate witnessbox command. Gate exit codes: 0 PASS, 1 FAIL, 2 ERROR. The machine-readable result for CI is gate.json in the results directory.

ebt-gate --installation prints the version and the SHA-256 of the gate and checker code; compare them with the release values in the decision log. Building from source, moving from the previous distribution and migration are covered in usage.

The documentation is written in Russian.

Guarantees and limits

The gate checks what was recorded and re-verifies the rule, but it is not a sandbox: the code under test and the gate run as the same OS user, so the guarantee is detection, not a privilege boundary (D26).

• A decision that was made but not recorded, with consequences not visible outside observation-level (OBS) scenarios, is not caught; mutation audit is a research tool, not part of the gate (D29).
• An installation tampered with before the run is not caught: the caller compares gate_sha256 and checker_sha256 with the release values (D33); in CI, actions/setup checks the wheel and --installation against the release (D45).
• A deliberate bypass from test.sh by the same user — a background process, ~/.gitconfig, PATH — is not caught (D26).
• A coordinated substitution of evidence and gate.json before the verdict is not caught; in CI the trusted stage re-checks the evidence and signs the result, but the evidence is still produced by the code under test (D34, D45).
• In CI, a verdict from the untrusted pull_request stage is not trusted: a workflow_run stage re-verifies the evidence against ebt/ from the base branch, signs gate.json and sets the witnessbox/gate commit status (D45, CI mode).

A human approves the specification (D4). The full "attack → how it is caught → what is not" table is in the threat model.

Documents

• How it works — one example from requirement to verdict, for a first look.
• Threat model — what the gate detects and what it does not promise.
• CI mode — two-stage GitHub Actions gate with a signed verdict and the witnessbox/gate status.
• New project — the ebt/ scaffold: specification, stack-specific test.sh and pinned.txt.
• Architecture — the actual modules and the verification flow.
• Extensions — public types and the contract for project checkers.
• Development — test commands, SDKs and the standalone build.
• Decisions — current status and the append-only decision log.

License

MIT. The SDK files (Ebt.java, ebt.ts, ebt.dart) are copied into application code unchanged and carry the line SPDX-License-Identifier: MIT.