invarlock 0.14.0

Run or import paired release-regression evidence. Verify it independently. Hand off a recipient-controlled decision.

Using independently supplied trust inputs, an artifact recipient can check whether one exact model derivative satisfies an agreed release-regression policy.

InvarLock is an open-source assurance engine for one paired baseline-versus-subject decision. It can execute both sides on the same deterministic schedule or import complete authenticated per-record material from another controlled evaluation. A closed request binds the artifacts, evaluation source or schedule, runtime and evaluator identities, scoring contract, and policy. InvarLock recomputes the permitted comparison and publishes a signed evidence bundle; a separate verifier replays that bundle against independently supplied trust anchors.

Evidence paths

Native execution and qualified import converge on the same signed evidence, independent verification, and reporting transaction:

For artifact delivery, the workflow continues from technical verification to a recipient-controlled decision:

authenticated evaluation → portable signed evidence → independent technical verification → recipient-controlled acceptance

An evaluation operator publishes the evidence; the artifact recipient controls the acceptance policy and decision.

invarlock evaluate request.yaml
invarlock verify evidence/
invarlock report evidence/

External evaluator adapters normalize source exports through the same versioned JSON, CLI, and Python qualification contracts. They remain in the example and integration layer; the core exposes evaluator-neutral contracts. The engine distinguishes three evidence paths:

Try the signed handoff locally

The service-free acceptance example runs a complete signed evaluation transaction over fixture artifacts and imported per-record results, independent technical verification, and recipient-policy handoff. It also demonstrates fail-closed rejection for changed artifacts, tampered evidence, untrusted or revoked signers, stale evidence, and contradictory envelope content.

git clone https://github.com/invarlock/invarlock.git
cd invarlock
python -m pip install -e .
make example-acceptance-handoff

The command runs from checked-in fixtures in a temporary workspace using the installed package on a regular CPU. See the offline handoff example.

Inspect published evidence

The repository carries strictly verified signed evidence packs across the built-in text runtime and first-party vision-text and TensorRT-LLM runtimes. Each uses a pinned public qualification suite and includes an independently signed verification receipt.

make public-evidence-audit
invarlock report \
  public_evidence/evidence/mistral-7b-weight-scale-hf/evidence
python -m json.tool \
  public_evidence/evidence/mistral-7b-weight-scale-hf/verification.receipt.json

Start with the public evidence index for the maintained inventory and interpretation limits. Reporting authenticates and explains the signed bundle. Acceptance remains a separate, recipient-controlled decision under current policy. Technical verification obtains the expected artifact, schedule, evaluated policy, runtime, evidence signer, and verifier anchors through channels independent of the submitted pack.

Run, verify, and report

Install the built-in Hugging Face provider for a native text comparison:

python -m pip install "invarlock[hf]"

For the native command below, start from the complete run request and replace every illustrative digest with one derived from the exact input. Import mode instead starts from the repository's schema-valid import request and omits --runtime-image and --runtime-image-digest; the request binds the complete imported provider sidecars.

invarlock evaluate request.yaml \
  --signing-key evidence-signer.pem \
  --runtime-image registry.example/invarlock-runtime@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa \
  --runtime-image-digest sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

invarlock verify artifacts/evidence-001/ \
  --trust-profile trust/trust-inputs.json \
  --receipt verification.receipt.json

invarlock report artifacts/evidence-001/ --html evidence.html --explain

evaluate --preflight --json returns the machine-readable result of complete execution-free validation. Native run mode delegates to a caller-authorized, digest-addressed Docker or Podman image. Import mode authenticates and replays complete provider sidecars and ordered per-record evidence locally. The getting-started guide and runtime-provider guide cover request construction, image preparation, device selection, host-only signing keys, and independently derived verifier inputs.

The release-regression decision

Both sides score the same authenticated records in the same order. InvarLock derives one of two built-in paired comparisons:

The conservative interval bound controls the policy; the point value remains descriptive. A policy may also require a minimum paired-record count and maximum interval width. Exact match includes an exact two-sided McNemar test; normalized NLL uses 2,048 deterministic paired schedule-resampling replicates. Normalized NLL specifically measures expected-continuation likelihood regression; broader model-quality claims require other evidence.

For task-specific deterministic scoring, a request may bind one authorized scorer extension. The extension derives one replayable value per record while the core retains ownership of pairing, aggregation, intervals, and policy. See schedule and policy for the full statistical contract and threshold guidance.

Import and qualify evaluator results

The core exposes one evaluator-neutral qualification contract as versioned JSON, the invarlock-qualify-evaluator companion CLI, and invarlock.engine.qualify_evaluator_export for Python callers. Open-source or proprietary evaluators reached through an SDK, CLI, or API normalize into that same boundary outside the core.

Verdict authority requires complete ordered per-record evidence that passes identity, provenance, schedule, and deterministic recomputation requirements. Aggregate-only outputs and unsupported judge results remain observation-only. A signed observation proves what was supplied; a replayable source additionally requires the identity, schedule, and recomputation guarantees above.

The maintained evaluator qualification matrix groups recognizable upstream evaluators by role and records their source version, evidence granularity, identity and provenance binding, replay status, and authority boundary. Each authoritative import demonstration starts with retained output from a pinned real model evaluation, passes through a source-shaped adapter, and completes the closed import replay. The matrix separately records model-running signed journeys. These are example-owned adapters and profiles; new profiles extend the same evaluator-neutral engine contract.

The examples/integrations/ directory contains maintained artifact-producing journeys for Hugging Face, PEFT, TorchAO, GGUF/llama.cpp, TensorRT-LLM, Hugging Face vision-text, and LM Evaluation Harness. The model-change workflow guide maps common model and runtime changes to native execution, optional-runtime, or import boundaries.

Hand off acceptance

The detailed signed verification receipt remains the replayable technical result. An optional in-toto/DSSE acceptance attestation transports that result and its exact subject binding. The artifact recipient still applies separate envelope and receipt trust, independent envelope and evidence freshness, contract-version, signer-status, and verdict policy.

Recipients can consume the acceptance envelope with a standalone reference verifier and maintained OPA/Rego or CUE policy configuration. The example authenticates the envelope and embedded receipt, then applies current recipient policy. Its conformance fixtures cover an accepted delivery, policy rejection, subject tampering, an untrusted signer, stale evidence, and an unsupported contract.

Acceptance-policy interoperability applies current recipient policy to the authenticated projection. invarlock verify performs complete evidence-pack replay. See the policy-engine interoperability reference.

Compatibility note: v0.13 evidence and receipts remain permanently verifiable and permanently ingestible as first-class dossier inputs. Every acceptance outcome remains controlled by the recipient's current policy.

Providers and diagnostics

Hugging Face Transformers is the built-in reference provider and supports both built-in metrics. First-party optional GGUF/llama.cpp, TensorRT-LLM, and Hugging Face vision-text packages are independently installable runtime integrations with their own dependency sets. The vision-text add-in supports exact-match comparisons over authenticated prompt and image parts. See runtime providers.

Spectral, random-matrix, and variance summaries live in the optional invarlock-diagnostics package. They are observation-only diagnostics. Their canonical JSON can be attached to the signed bundle and appears in a separate report section. The selected paired comparison and policy remain the sole technical-verdict inputs. See diagnostics.

Documentation

• Run and review: getting started, evaluation requests, schedule and policy, and evidence and verification.
• Understand the claim: assurance case, decision semantics, and trust model.
• Integrate: CLI, contracts, acceptance attestations, compatibility covenant, evaluator qualification, policy-engine interoperability, runtime providers, and Python API.

InvarLock is pre-1.0. Canonical artifact formats carry explicit format versions; the Python embedding facade may evolve between minor releases.

Questions and design discussions belong in GitHub Discussions. Report bugs through GitHub Issues and security concerns through SECURITY.md.

If you ship or receive derived model artifacts and want to co-publish a real evidence-and-receipt handoff using the open engine, start a design-partner discussion.

Apache-2.0 — see the license.