shipwright-check 0.1.0

Pre-ship reliability check for an AI build's claims, docs, and citations — composes verity, firewall, and grounded over your own project (bring-your-own, consent-first). Checks what your AI ships in text, not its runtime behavior.

Shipwright

Catch it before your users do. A pre-ship reliability check for your AI build's claims, docs, and citations — run it on your own project, privately, before you ship.

It composes three deterministic verification gates over the text your build produces and ships:

Check	What it catches	Powered by
Doc/output over-claims	factual claims in your README/output that contradict your declared ground truth	firewall
Result-claim hygiene	result-claims (accuracy/win-rate) with no sample size, no holdout, or possible leakage	verity
Citation grounding	cited claims whose source doesn't actually support them (fabricated/mismatched citations)	grounded

It reports a transparent 1–5 reliability level and names each unearned point next to the fix.

Honest scope (what it does and doesn't do)

Shipwright checks what your build claims and ships in text. It does not test your code's runtime behavior, and a 5/5 means these specific deterministic checks pass — not "provably reliable AI." It runs only on the path you give it and sends nothing anywhere (consent-first). A gate that isn't installed, or a check with no matching content, is reported as n/a — never a fake pass. MCP/tool authorization is deliberately not a dimension: our mcp-bench is a benchmark of scanners, not a scanner you point at your repo, so claiming an MCP-authz check here would be an over-claim of our own.

Install

Shipwright itself has no dependencies. Install it and the three gates (the gates are published from the StellarRequiem repos, not under their generic PyPI names):

pip install -e .                                              # shipwright (this repo)
pip install git+https://github.com/StellarRequiem/verity-core \
            git+https://github.com/StellarRequiem/firewall \
            git+https://github.com/StellarRequiem/grounded

Use

shipwright init                         # scaffold a starter truth.yaml + a CI workflow
shipwright check .                      # check the current project
shipwright check . --truth truth.yaml   # declare your ground truth for the over-claim check

Writes shipwright-report.md and prints the level. Exit code is CI-gateable: 0 pass · 1 review · 2 refuse. Each gap is linked to the lesson that closes it, in the Building Verifiable Software course.

Agents can self-check over MCP: verity-mcp (installed with verity-core) is a Model Context Protocol server — point your agent at it to gate its own claims.

Status

Early (v0.1.0), single author, no users yet — the leading hypothesis is that teams ship LLM apps whose claims outrun what they can support, and there's no cheap, honest, local way to catch that before users do. This is the free, local tier; a hosted progress/badge surface is a later build.