prism-verify 1.5.0

Runtime adjudication service for agent workflows. Family-different, reasoning-stripped, multi-lens verification with replayable receipts.

日本語 | 中文 | Español | Français | हिन्दी | Italiano | Português (BR)

Runtime adjudication service for agent workflows. Family-different, reasoning-stripped, multi-lens verification with replayable receipts — for code, tool-calls, citations, and response sycophancy. Landing page & handbook →

Install

Install the prism CLI (and the HTTP service) on your PATH:

uv tool install prism-verify        # or: pipx install prism-verify

Zero Python? Use the npm launcher (downloads + SHA256-verifies a prebuilt binary):

npx @mcptoolshop/prism-verify verify --artifact @file.py --intent "..." --caller-family openai

Or add it as a library — extras: [anthropic] [openai] [google] [mcp] [http] [all]:

uv add prism-verify
# or
pip install "prism-verify[all]"

Quick start

Prism always verifies with a model family different from the caller's (Lock 1), so configure at least one alternate-family provider. Generate an Ed25519 signing key (the default — receipts are verifiable by anyone with the public key) so receipts can be written, or use PRISM_DEV=1 for local play:

prism keygen --out ~/.prism/signing_key.pem      # Ed25519 keypair (default signing)
export PRISM_SIGNING_KEY=~/.prism/signing_key.pem # or: export PRISM_DEV=1 (local play)
export ANTHROPIC_API_KEY="sk-ant-..."             # alt-family verifier for an OpenAI-family caller

prism verify \
  --artifact @myfile.py \
  --intent "Sort a list in O(n log n)" \
  --caller-family openai \
  --provider anthropic

Legacy alternative: export PRISM_SIGNING_SECRET="$(openssl rand -hex 32)" signs receipts with HMAC instead (verifiable only by holders of that shared secret — see Receipts).

Architecture

Prism enforces four architectural locks at the API contract:

1. Family-different — caller's model family is always excluded from verification
2. Reasoning-stripped — producer CoT is stripped before crossing the family boundary
3. Multi-lens — at least 3 independent lenses run in parallel
4. Submodularity-aware — refuses if lenses agree too much (collapsed signal)

For citation artifacts, a layered floor runs before the LLM groundedness lens — each deterministic stage refuses what it can prove, else abstains:

• Existence floor — live arXiv/Crossref retrieval; a fabricated identifier is dropped, not reasoned about.
• Numeric/unit floor — a percentage swap, a unit-scale slip (42 milli- vs micro-arcsec), or a comparison-direction falsehood (5.0 < 5.8 ≠ "exceeded") is caught arithmetically.
• Groundedness lens — the family-different, reasoning-stripped LLM check against the retrieved abstract.
• Orthogonal NLI floor (opt-in, PRISM_NLI_FLOOR) — an encoder NLI cross-encoder vetoes a "supported" the LLM gave but a mechanistically-different model does not corroborate.

Bring your own verifier

The groundedness lens can run against a model you host instead of a hosted API — opt-in via PRISM_LOCAL_VERIFIER_ENDPOINT, family-different, and fail-open to your hosted verifiers. The most frequent check costs nothing per call and your evidence stays local. An opt-in capture sink (PRISM_HARVEST_PATH) records the (claim, evidence, verdict) triples so you can train one. See the handbook.

Sycophancy (response verification)

Beyond code, tool-calls, and citations, prism judges a model RESPONSE for regressive sycophancy — telling the user what they want to hear over what is correct (affirming a false premise, abandoning a correct answer under mere pushback). It runs a family-different, reasoning-stripped fine-tuned specialist as the sycophancy lens — opt-in via PRISM_SYCOPHANCY_ENDPOINT, fail-open to abstain (never a silent "not sycophantic"). Agreement with a correct user, or conceding to a well-evidenced rebuttal, is faithful, not sycophantic. See the handbook.

Calibration & benchmark (prism eval)

prism is built to be measured, not just asserted. prism eval runs the lenses over a labeled corpus and reports — on prism's own data — per-lens precision/recall/MCC, the inter-lens diversity matrix (Krippendorff α + pairwise Cohen κ), submodular coverage-gain, verdict accuracy, and confidence calibration (ECE/Brier), each with an honest confidence interval.

prism eval --split all --runs 1        # measure against the bundled corpus (needs a verifier)
prism eval --offline                    # deterministic mock (CI smoke; NOT a real measurement)

Measured, not asserted (latest run → — local Ollama mistral-small:24b, 111 samples, 2026-06-14):

What we measured	Result
Per-lens MCC (contract / cross_boundary / invariant / groundedness)	0.33 / 0.71 / 0.48 / 0.78
Lens independence (Krippendorff α) — but contract↔invariant is redundant (Cohen κ)	α 0.162 · κ 0.717
Overall verdict accuracy / calibration (ECE)	0.667 / 0.241
Honest contamination check: contaminated vs uncontaminated accuracy	0.560 vs 0.754 (Δ 0.194)
Citation safety: a bad citation is never blindly accepted (off-accept recall)	1.000

The corpus is small and the headline findings are directional — but they are real measurements, not mock, and the report is candid about what they do and don't show (e.g. coverage gain is 0 on this invariant-dominant corpus). See the evaluation handbook for the method and a worked example.

HTTP service

Run prism as an HTTP service (needs the [http] extra):

prism serve --host 127.0.0.1 --port 8000      # OpenAPI docs at /docs

Endpoint	What it does
POST /verify	Verify an artifact (same contract as the CLI). Blocks within the budget; Prefer: respond-async + a webhook URL → 202, verdict delivered to the (signed) webhook.
GET /replay/{receipt_id}	The signed receipt + signature_valid.
POST /verify-receipt	Verify a standalone receipt (cross-tool).
GET /healthz	Liveness + configured verifier families (no auth).

Set API keys (hashed at rest) — prism is fail-closed, so /verify is refused until keys are configured or you opt into local no-auth:

export PRISM_API_KEYS="<sha256(key1)>,<sha256(key2)>"   # callers send: Authorization: Bearer <key>
export PRISM_WEBHOOK_SECRET="<random>"                  # to sign async/escalate webhook deliveries
# local dev only:
export PRISM_HTTP_ALLOW_NO_AUTH=1

Errors are RFC 9457 application/problem+json; POST /verify honours an Idempotency-Key header and a per-key rate limit (429 + Retry-After). Async/escalate webhooks are Standard-Webhooks-signed, SSRF-guarded (no internal/metadata targets), retried, and carry a named cancel-event compensator.

Receipts & signing (Ed25519, verifiable by anyone)

Every verification produces a signed, replayable receipt in ~/.prism/receipts.db. v0.4 signs new receipts with Ed25519 (RFC 8032) by default, so a different tool can verify a prism receipt with prism's public key — no shared secret:

prism keygen --out ~/.prism/signing_key.pem    # generate an Ed25519 keypair
export PRISM_SIGNING_KEY=~/.prism/signing_key.pem
prism pubkey                                    # publish this public key + kid to consumers

# a consumer (e.g. role-os) verifies a receipt with ONLY the public key:
prism verify-receipt receipt.json --public-key prism-pub.pem

The signature covers the verdict, the pre/post-strip artifact hashes, the verifier model, the submodularity matrix, the per-lens prompt hashes (byte-for-byte replayable), the citation retrieval pins, and the signing alg/kid. Legacy HMAC receipts still verify (set PRISM_SIGNING_SECRET); PRISM_DEV=1 mints a dev key for local play. Prism refuses to start the verify / replay / serve / MCP paths if no key is configured, rather than silently signing with a publicly known key.

Manage stored receipts with the compensator commands:

prism receipt delete <receipt_id>
prism receipt prune --older-than 90d --yes

Security & privacy

• Threat model. Prism reads the artifact + intent you pass and the verifier models' responses, and writes signed receipts to a local SQLite DB. It does not read your source tree, environment, or credentials beyond the provider API keys you supply via environment variables. Receipt signatures give third-party verifiability (Ed25519: a consumer verifies with the public key, no shared secret) but are not tamper-proof against a local-root attacker who can read the on-disk private key — that's the same ceiling as the HMAC secret. For genuine tamper-resistance, hold the key in an HSM and anchor receipts in a transparency log (the named hardening path).
• HTTP surface. prism serve binds loopback by default, is fail-closed (no /verify without API keys), hashes keys at rest, and SSRF-guards caller-supplied webhook URLs (no internal/link-local/metadata targets). It runs caller-supplied artifacts through a model; an artifact may attempt prompt injection but cannot change the verdict schema or exfiltrate prism's provider keys.
• No telemetry. Prism sends requests only to the model providers you configure (Anthropic / OpenAI / Google / local Ollama). Nothing else.
• Full policy: SECURITY.md.

License

MIT