evalopt-graph 0.1.0

evalopt turns host observations into one deterministic, content-addressed acceptance decision. It does not run models, choose tools, edit repositories, or pretend that retrieved text is true. Your agent runtime does the work; the kernel decides whether the resulting evidence satisfies an explicit policy.

Install

pip install evalopt-graph

Python 3.10–3.14 is supported. The stable kernel has no required runtime dependencies.

Decide, serialize, replay

from evalopt_graph import AcceptanceDecision, AcceptanceInput, GovernancePolicy, evaluate_acceptance

policy = GovernancePolicy(required_gates=("tests", "lint"))
observed = AcceptanceInput(
    observed_at="2026-07-20T12:00:00+00:00",
    gate_results=(("tests", "PASS"), ("lint", "PASS")),
)

decision = evaluate_acceptance(policy, observed)

assert decision.status == "ACCEPTED"
assert decision.reasons == ("policy_satisfied",)
assert decision.validate()

# Decisions are plain, content-addressed records.
stored = decision.to_dict()
restored = AcceptanceDecision.from_dict(stored)
assert restored.replay(policy, observed)

The evaluator is pure: the same policy and input produce the same decision. Terminal outcomes are ACCEPTED, BLOCKED, UNVERIFIED, UNSUPPORTED, and FAILED, each with stable reason codes and hashes for the policy, input, evidence records, and decision.

Stable kernel API

The package root exports exactly ten stable symbols in v0.1.x:

from evalopt_graph import (
    GovernancePolicy,
    ClaimRecord,
    EvidenceRequest,
    EvidenceMaterial,
    EvidenceAttestation,
    SupportAssessment,
    EvidenceAuthority,
    AcceptanceInput,
    AcceptanceDecision,
    evaluate_acceptance,
)

• GovernancePolicy declares required gates, trust, freshness, evaluator, claim, verifier, and record authorization requirements.
• EvidenceAuthority applies controller-owned adapter policy to material already retrieved by a host.
• EvidenceAttestation binds source policy, retrieval metadata, selected content, and hashes.
• SupportAssessment separately records whether selected content supports, contradicts, or is insufficient for a claim.
• evaluate_acceptance evaluates a complete immutable observation once and returns a replayable AcceptanceDecision.

See kernel.py for the compact public contract and test_kernel.py for complete evidence-authority examples.

Kernel owns / host owns

This boundary is intentional. Importing evalopt_graph loads the kernel without importing provider SDKs, the compatibility graph, CLI code, filesystem adapters, benchmark adapters, or optional frameworks.

Evidence, not a truth oracle

Evalopt keeps three questions separate:

1. Was this exact material retrieved under an allowed adapter policy? An attestation binds the retrieved bytes, selected span, source class, time, parser identity, and policy.
2. Does the selected material support this exact claim? A support assessment records the verifier and its result independently of retrieval integrity.
3. May these exact records authorize acceptance? Governance policy allowlists the authority policy, verifier, and evidence records before the decision can use them.

That structure makes evidence tampering and unsupported load-bearing claims visible. It does not prove semantic truth, deployed behavior, source correctness, or model capability.

Current evidence ladder

• IMPLEMENTED / UNIT_PROVEN: pure API, immutable records, distinct terminal states, authorization, serialization, replay, and import isolation.
• INTEGRATION_PROVEN: the compatibility host and optional Harbor verifier mapping terminate through the same kernel decision path.
• CONFORMANCE_PROVEN: deterministic generated and authored cases exercise a bounded evidence boundary mechanism.
• Not established: benchmark superiority, external comparison, independent reproduction, or SOTA.

Read the evidence report and prospective external protocol for the claims, controls, and gaps. There is no live-model governance campaign or official Docker SWE-bench result in this release.

Integrate it anywhere

The kernel is host-independent: adapt observations from Codex, Claude Code, OpenHands, LangGraph, CI, or your own runtime into AcceptanceInput, then store the returned decision beside the policy and input used to create it.

An optional Harbor 0.18 conformance task demonstrates verifier-side mapping without importing Harbor into the kernel. It validates integration wiring only; it is not an external benchmark.

The historical graph, standalone CLI, provider clients, Codex bridge, research loop, and filesystem adapters remain as deprecated compatibility surfaces for one migration cycle. They are not stable API or policy authorities. New integrations should depend only on the ten root exports above.

Documentation

• Architecture and trust boundaries
• Evidence results and limitations
• Prospective benchmark protocol
• Harbor integration
• v0.1.0 release notes
• Changelog

Community and security

Bug reports, focused proposals, documentation improvements, and integrations are welcome. Start with the contribution guide and follow the Code of Conduct.

Please report vulnerabilities privately using GitHub Security Advisories, not a public issue. See the security policy for supported versions and response expectations.

Development

git clone https://github.com/ajaysurya1221/evalopt-graph.git
cd evalopt-graph
uv venv .venv
source .venv/bin/activate
uv pip install -e ".[dev]"

python -m compileall -q src tests scripts
python -m pytest -q
python -m ruff check src tests scripts
python -m ruff format --check src tests scripts
bash scripts/smoke_test.sh
uv build

License

Released under the MIT License. Copyright © 2026 Ajay Surya Senthilrajan.