synapsek-claims (svc)

Verified claims for AI artifacts. A framework-agnostic, cryptographically verifiable layer that signs measured quality claims about any AI artifact or pipeline — and lets a buyer re-derive them with one command.

You have seen vendor-reported benchmark numbers. You may have re-run one and found it did not come out the same. svc is the missing referee:

• the issuer signs a claim document: measured values, the exact evidence traces, the exact artifacts and code used, the pinned environment;
• the buyer runs one command that (1) checks the signature against a trusted key, (2) re-hashes every pinned file, (3) re-checks every evidence trace (sha256 + Merkle), (4) recomputes every claimed number from the evidence, optionally re-executes the pinned evaluation, and (5) re-derives every pass/fail status from the documented thresholds.

If the issuer claimed more than the numbers support, verification fails. That last property — anti-overclaiming by construction — is what makes this different from a signed eval dashboard.

$ svc verify claim.json --trusted-pubkey issuer.pub --base-dir . --rerun
=== SVC VERIFY synapsek.claim/knowledge-encoder-dedup/v3b-seed1 ===
  [PASS] L1 authenticity: valid
  [PASS] L2 integrity: 10 files match
  [PASS] L3 evidence: 2 traces match
  [PASS] L4 soundness: 1/1 values recomputed
  [PASS] L4b rerun: 1 claims re-executed
  [PASS] L5 claims: statuses + recommendations re-derived
  claim dedup_spearman: value=0.4172424238214698 recomputed=0.4172424238214698 status=proven [OK]
RESULT: PASS — all layers verified; claims re-derived

What a claim is

A claim is a JSON document:

• subject — what is being claimed about: files (hashed) + evidence traces (hashed + Merkle-rooted);
• pinned — the environment, dataset, and config the measurement used;
• claims — each has a metric, a measured value, a documented threshold, a status (proven / not_proven), and a derivation (a rule that tells the buyer how to recompute the value from the evidence);
• recommendation — which labels are recommended (only the proven ones may be) and which are explicitly not;
• disclosures — honest limitations;
• signature — Ed25519 over the canonical form of everything above.

Derivation rules are a tiny grammar, no arbitrary code in rules:

script: is the only rule that executes code — it is the issuer's pinned evaluation script, fully visible to the buyer, and it only runs if the buyer passes --rerun (or explicitly accepts it). The buyer reads the script before it can ever run.

Quickstart

pip install synapsek-claims        # or: pip install -e .

# buyer side
svc verify examples/pipeline-dedup/claim.json \
    --trusted-pubkey examples/trusted-issuer.pub \
    --base-dir examples/pipeline-dedup \
    --rerun                          # also re-execute the pinned eval (CPU)

# issuer side
svc keygen --out-dir ./keys         # new attestation authority
svc attest skeleton.json --key keys/issuer_ed25519 --base-dir ./claimdir
svc registry add registry/registry.json --claim claimdir/claim.json --issuer-id me

Exit code 0 = PASS, 1 = FAIL. --json gives a machine-readable report.

The two working examples

Both are real, measured, and verify PASS as shipped:

1. examples/pipeline-dedup — a knowledge encoder (2-layer transformer + Perceiver resampler, K=128 latents). Claims Spearman 0.4172 between latent similarity and token Jaccard over a pinned 100-doc holdout (4950 pairs, full pair-level evidence trace). --rerun re-executes the pinned eval on CPU and reproduces the value bit-exactly.
2. examples/cartridge-v11 — the Synapsek knowledge cartridge v1.1, wrapped through the same pipeline as its full attestation. 13 claims: the 4 v0 gist dimensions recorded as not_proven (below pre-registered bars — an honest claim is allowed to say "no"), and 9 QA-fidelity numbers re-derived from the pinned per-item traces, including cross-family results (a 7B model decoding latents produced by a small encoder).

What this is NOT

• Not a model-signing tool. OpenSSF/Sigstore-style integrity signing is a commodity and a good one; svc is complementary. It signs quality claims and proves they re-derive, not just that the bytes did not change.
• Not a benchmarking service. We pin your evaluation. svc cannot make a bad number look good, and it is structurally hostile to one.
• Not a quality certifier. A proven claim means: this measured value is reproducible from the pinned evidence at/above the documented threshold. The threshold is the issuer's documented commitment, and the buyer is free to think it is low.

Security model

• Ed25519 over deterministic canonical bytes (sorted keys, compact JSON).
• Only the public key ships with claims; the private key stays with the issuer. Buyers verify against a trusted public key held out-of-band; issuer.public_key must equal the trusted key (no rogue self-issuance).
• The registry is a trust anchor: claim id → issuer key → claim hash.
• Every layer is independently checkable; the report says which layer failed and why.

Tests

python3 -m pytest tests/ -q     # 16 tests: roundtrip, tamper L1/L2/L3/L4b/L5, registry

Layout

src/svc/            canonical, crypto, schema, rules, rerun, claim, verify, registry, cli
tests/              16-test suite
examples/pipeline-dedup/   real claim, rerunnable on CPU (artifact/encoder.pt is a release asset)
examples/cartridge-v11/    real claim wrapping the v1.1 attestation + QA traces
docs/DESIGN.md      schema, threat model, design decisions
RELEASE-CHECKLIST.md  10-minute publish handoff

License

Apache-2.0. See LICENSE.