underwrite
underwrite turns local evaluation artifacts into a common observation, computes
measurements under an explicit policy, and classifies change candidates against
declared evidence. It keeps missing evidence visible: not_measured,
indeterminate, and abstained are distinct outcomes.
It is a Python CLI and library. Artifact inspection checks bytes and declared hashes; release checks examine declared linkage and record presence. These checks do not establish independent provenance or grant deployment permission.
Install from source
Requires Python 3.12 or later and uv.
git clone https://github.com/ntts9990/underwrite.git
cd underwrite
uv sync --frozen --all-groups
uv run --frozen --no-sync underwrite --help
The repository includes the underwrite-core workspace package. No adjacent
checkout is needed to install or use the example workflows.
PyPI release preparation
The functional 0.0.1 release is prepared for underwrite-core and underwrite;
publication is not yet confirmed. The existing underwrite 0.0.0 entry is a
reserved placeholder. Use the source installation above until both 0.0.1
releases are available. After publication, the planned CLI installation is:
uv tool install 'underwrite==0.0.1'
underwrite --help
Maintainers: create the GitHub environment pypi, restrict it to main, and
configure a PyPI GitHub Trusted Publisher on each project with owner ntts9990,
repository underwrite, workflow filename publish.yml, and environment pypi.
For the new underwrite-core project, register a pending publisher under that
project name; add the publisher to the existing underwrite project in its
Publishing settings. No stored PyPI API token is required.
Once configured, run Publish PyPI manually from the main branch in
GitHub Actions. It tests both packages, builds and validates the exact
distributions, and installs the wheels in a clean
environment before publishing core and then the application. Only the separate
publish job receives OIDC permission. Retry with the same artifacts after a
partial upload; existing registry files must match exactly. A changed artifact
requires a new version. Confirm both PyPI project releases and a fresh indexed
installation before describing the release as available.
The workflow follows the official uv publishing guide and PyPI Trusted Publisher guide.
Normalize an artifact
Choose a format and exact supported version explicitly:
| Producer | --format |
--version |
|---|---|---|
| DeepEval | deepeval.test-run |
4.1.1 |
| Inspect AI | inspect.eval-log |
0.3.263 |
| Langfuse | langfuse.observations-v2 or langfuse.scores |
4.35.0 |
| OpenInference | openinference.traces |
0.1.38 |
| OTLP JSON | otlp-json.traces |
0.160.0 |
| promptfoo | promptfoo.eval-output |
0.123.0 |
| Native evidence bundle | underwrite.evidence-bundle |
v1 |
These are bounded payload profiles, not compatibility claims for every export
from those producers. project is an explicit alias of ingest.
mkdir -p output
uv run --frozen --no-sync underwrite ingest \
--format underwrite.evidence-bundle --version v1 \
--file fixtures/examples/measurement/bundle.json --json \
> output/observation.json
uv run --frozen --no-sync underwrite measure \
--observation output/observation.json \
--policy fixtures/examples/acceptance/policy-quality.json --json \
> output/read-quality.json
uv run --frozen --no-sync underwrite measure \
--observation output/observation.json \
--policy fixtures/examples/acceptance/policy-latency.json --json \
> output/read-latency.json
The native bundle declares schema_version: underwrite.evidence-bundle.v1 and
carries readings, source claims, and availability records. An observation keeps
those source claims separate from the measurement. Measurement requires a
complete policy identifying inputs, subject, calibration, quorum, and statistical
thresholds. A successful command exit means the read was produced; inspect the
read's state before drawing a conclusion.
--http-body-file FILE reads an already captured HTTP body from a local file;
it does not fetch a URL. Optional --source-ref and --source-locator describe
the supplied source. When no source reference is given, byte identity supplies
a content hash, not an independently verified origin. See command --help for
input size and depth limits.
Malformed native input produces a typed error on stderr and no observation on stdout. Try a bundle with an empty run identifier:
mkdir -p output
printf '%s\n' '{"schema_version":"underwrite.evidence-bundle.v1","run_id":"","sources":[],"readings":[],"availability":[]}' \
> output/malformed-bundle.json
if uv run --frozen --no-sync underwrite ingest \
--format underwrite.evidence-bundle --version v1 \
--file output/malformed-bundle.json --json \
> output/malformed.stdout 2> output/malformed.stderr; then
ingest_exit=0
else
ingest_exit=$?
fi
printf 'exit: %s\n' "$ingest_exit"
wc -c < output/malformed.stdout
cat output/malformed.stderr
This reports exit 1, stdout size 0, and this stderr JSON:
{"schema":"instrument_error.v1","command":"ingest","code":"INVALID_INPUT","reason":"MALFORMED_EVIDENCE_BUNDLE_PAYLOAD","exit_code":1}
Classify a candidate
accept binds a candidate to declared claims and policy-matched reads. It reports
whether the supplied evidence supports the requested acceptance basis and retains
uncertainty or missing measurements in the decision.
uv run --frozen --no-sync underwrite accept \
--candidate fixtures/examples/acceptance/candidate.json \
--claims fixtures/examples/acceptance/claims.json \
--read output/read-quality.json \
--read output/read-latency.json --json
Repeat --read for each required claim. Exit 0 means a classification was
produced, not approval. This example produces screened, requires human review,
and keeps merge_authorized false. The inputs are synthetic local examples;
the reads above are computed from the supplied bundle and policies.
Other checks
inspect CASE.json --json: inspect one local artifact manifest and its projection.check RELEASE.json --json: check a release manifest's declared evidence linkage.absence --root DIR --glob '*.json' --min 1 --json: require a nonempty population.boundary scan DIR --json: scan JSON and JSONL strings for configured verdict vocabulary.drift check --pins-root PINS --siblings-root SOURCES --json: compare a user-supplied pin registry against local source checkouts.pins refresh --repo NAME --pins-root PINS --siblings-root SOURCES: refresh a configured repository's pins from local bytes.
Pin commands are generic utilities; no preconfigured source registry is included.
Allowlist and exemption evaluation requires an explicit --now when applicable.
Malformed inputs and missing paths remain configuration errors rather than passes.
Use each command's --help for its options and exit behavior. JSON contracts live
in contracts/; each command family has its own output and error schema.
Development and license
See CONTRIBUTING.md for the checks and contribution process, SECURITY.md for private vulnerability reporting, and AGENTS.md for technical boundaries. The core package uses only the Python standard library; application adapters own local I/O and schema validation.
underwrite is licensed under Apache-2.0. LICENSING.md records the adapted policy templates' CC licenses.
Metadata
Release files for underwrite 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| underwrite-0.0.1.tar.gz | 137.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| underwrite-0.0.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 307.7 kB
Release files / underwrite-0.0.1.tar.gz
| Download URL | underwrite-0.0.1.tar.gz |
|---|---|
| Size | 137.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
85cdc103d1bb2786eff2f204f0decb6fd135852fc8d9acdaabf205fea1d3c294
|
|
BLAKE2b-256 checksum How to use checksums |
d1bf342cb80f00ea4a8cb20f07a3b1d76464ec6f2ede3ab5a2e07736570d6c49
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.29 {"installer":{"name":"uv","version":"0.11.29","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / underwrite-0.0.1-py3-none-any.whl
| Download URL | underwrite-0.0.1-py3-none-any.whl |
|---|---|
| Size | 170.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b448244313eb8f0606048306a1a6f729215faa1a9c5b451964588f5a5ae92051
|
|
BLAKE2b-256 checksum How to use checksums |
62502b2e165340c50697c5ea48ac65ffb5cb75d40f1d35c6586dc779b6b2179b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.29 {"installer":{"name":"uv","version":"0.11.29","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|