astraform-remote-domain-conformance

This is the reusable Python conformance harness for remote-domain.v1.

Brutal truth: if every partner team needs a platform engineer on Zoom to prove their service is valid, you do not have a platform. You have consultancy with a protocol document attached.

This package exercises the real lifecycle:

• manifest
• prepare
• status
• execute-work
• inspection
• shutdown

And it validates payloads against the versioned schema from the pinned remote-domain.v1 release bundle, not just happy-path demo assertions.

It also includes a lightweight remote Policy Wind Tunnel provider conformance path for /policy-wind-tunnel/... routes. That path checks pack metadata, domain/report labels, outcomeSchema, CEL-compatible policyExpressions, presets, run creation, status, lifecycle control, the generic policy_wind_tunnel_bundle.v1 envelope, artifact, and evidence-pack availability. A weak bundle that only returns schemaVersion and runId is not conformant; it must expose the domain, pack, preset, branches, metrics, segments, timeline, decision gates, artifacts, and domain payload that the dashboard and outcome report can render without custom frontend code.

Install

The published PyPI baseline is 0.1.6. This source tree is the 0.1.7 release candidate, which adds strict Policy Wind Tunnel scorecard-role conformance against contract 1.0.4.

pip install astraform-remote-domain-conformance==0.1.6

Workspace release target:

• astraform-remote-domain-conformance==0.1.7

For repository-local conformance development beyond the published 0.1.6 baseline, use an editable install or a private pre-release wheel.

Repo-local development:

pip install -e './remote-domain-conformance-python[test]'

Run Against A Live Service

remote-domain-conformance \
  --base-url http://localhost:8092 \
  --domain-id my-domain

Require Population Builder catalog conformance:

remote-domain-conformance \
  --base-url http://localhost:8092 \
  --domain-id my-domain \
  --population-catalog

Validate a catalog file before the provider serves it:

remote-domain-conformance \
  --domain-id my-domain \
  --population-catalog-file ./population_catalog.json

Run Against A Local ASGI App

remote-domain-conformance \
  --app my_remote_domain.main:app \
  --domain-id my-domain

Baseline conformance performs the published v1 lifecycle once. To prove the stronger async worker transport/effect boundary, pass an explicit out-of-band profile:

remote-domain-conformance \
  --app my_remote_domain.main:app \
  --domain-id my-domain \
  --opportunity-worker-profile conformance/opportunity-worker-profile.json \
  --provider-artifact-digest "sha256:<64 lowercase hex characters>" \
  > build/conformance-run-report.json

Extract the nested opportunityWorkerConformanceReport, then sign that file in a separate protected job which runs no provider or repository code:

remote-domain-conformance \
  --sign-conformance-report build/opportunity-worker-conformance-report.json \
  --opportunity-worker-profile conformance/opportunity-worker-profile.json \
  --attestation-private-key /secure/release-ed25519-private-key.pem \
  --attestation-key-id partner-release \
  --attestation-output build/opportunity-worker-attestation.dsse.json

The protected signer recomputes the supplied profile digest and requires exactly one matching passing result for every classified tool. The signed conformanceReportDigest covers the report's passing baseline lifecycle, cross-attempt replay, and per-tool probe outcomes, plus its exact OCI subject binding. The public report schema is shipped in the opportunity-worker contract profile. Release automation must fail when the protected signer is unavailable; an ephemeral-key fallback is not a release proof.

The profile schema version is remote_domain_opportunity_worker_conformance_profile.v1; it contains exact toolEffects, one safe probe per tool, and optional personaConfiguration. It is not part of the remote-domain.v1 manifest schema.

Worker-profile responses are consumed from response.content, not decoded response text. Before a passing report can exist, the runner requires one exact JSON Content-Type, strict UTF-8, valid Unicode scalar text without U+0000, unique object keys, one JSON value, and lossless I-JSON/RFC 8785 numbers. Signed worker proof binds protocol responses to the manifest-advertised media type; the historical application/json fallback remains baseline-v1 compatibility only.

Run Policy Wind Tunnel Provider Conformance

remote-domain-conformance \
  --wind-tunnel \
  --base-url http://localhost:8092 \
  --pack-id my-domain-policy-pack \
  --preset-id conformance-proof

When the pack advertises opportunityGenerationFinalizationMode=PLATFORM_OWNED_V2, the wind-tunnel runner automatically executes a clock-driven V2 probe. It pins the immutable provider revision, cancels a generated run, finalizes it with a stable operation ID, validates the strict receipt schema, downloads and hashes every declared artifact, proves an exact replay, and rejects contradictory reuse of the operation ID.

V2 conformance requires a ScenarioLabFinalizationV2ProbeDriver backed by real OS processes. The driver supplies a started subprocess.Popen handle bound to the provider base URL under test, exposes the durable PREPARED receipt and artifact bytes, terminates the original provider, and launches a distinct process against the same durable state. The harness independently verifies that the old process exited and its endpoint is unavailable before accepting the replacement PID, then requires the replacement process to publish the byte-equivalent canonical /pack identity. Legacy V1 coverage is provisioned out of band as recovery state; the harness never launches a fresh PLATFORM_OWNED_V1 run. The driver also runs two distinct child processes against the configured receipt/artifact store so the harness can verify lock exclusion, cross-process visibility, and atomic publication. Both the successful and terminal-precedence paths cross a real process restart. Completion delivery is failed once deliberately and must retry independently without another FINALIZE request. The harness also injects each published artifact and receipt fault through the driver and verifies rejection through the provider HTTP boundary; an in-process ASGI app swap or a relabelled process is not restart proof.

Python Usage

import asyncio

from astraform.remote_domain.conformance.runner import ConformanceScenario
from astraform.remote_domain.conformance.runner import OpportunityWorkerConformanceProfile
from astraform.remote_domain.conformance.runner import PolicyWindTunnelConformanceScenario
from astraform.remote_domain.conformance.runner import ToolProbe
from astraform.remote_domain.conformance.runner import run_conformance
from astraform.remote_domain.conformance.runner import run_opportunity_worker_conformance
from astraform.remote_domain.conformance.runner import run_policy_wind_tunnel_conformance


report = asyncio.run(
    run_conformance(
        base_url="http://localhost:8092",
        scenario=ConformanceScenario(domain_id="my-domain"),
    )
)

print(report.to_dict())

worker_report = asyncio.run(
    run_opportunity_worker_conformance(
        base_url="http://localhost:8092",
        scenario=ConformanceScenario(domain_id="my-domain"),
        profile=OpportunityWorkerConformanceProfile(
            tool_effects={"lookup_case": "READ_ONLY"},
            tool_probes=(ToolProbe("lookup_case", {"caseId": "case-1"}),),
        ),
        provider_artifact_digest="sha256:<64 lowercase hex characters>",
    )
)

wind_tunnel_report = asyncio.run(
    run_policy_wind_tunnel_conformance(
        base_url="http://localhost:8092",
        scenario=PolicyWindTunnelConformanceScenario(
            pack_id="my-domain-policy-pack",
            preset_id="conformance-proof",
        ),
    )
)

print(wind_tunnel_report.to_dict())

Publishing Status

Release validation starts with the standalone package tests:

pytest remote-domain-author-kit-python/tests remote-domain-conformance-python/tests

PyPI artifacts are immutable. Do not rerun a publish for an already published version; run smoke-only verification or bump the SDK version.

Public package note: PyPI distributions expose this SDK implementation. Keep host runtime internals and domain-private logic out of this package.