astraform-remote-domain-conformance

This is the reusable Python conformance harness for remote-domain.v1.

Brutal truth: if every partner team needs a platform engineer on Zoom to prove their service is valid, you do not have a platform. You have consultancy with a protocol document attached.

This package exercises the real lifecycle:

• manifest
• prepare
• status
• execute-work
• inspection
• shutdown

It validates payloads against the exact embedded remote-domain.v1 contract release resolved automatically during package builds. The package includes its release lock and provenance under _contract/contracts/; a LOCAL_PRERELEASE snapshot is distinguishable from the published baseline.

It also includes a lightweight remote Policy Wind Tunnel provider conformance path for /policy-wind-tunnel/... routes. That path checks pack metadata, domain/report labels, outcomeSchema, CEL-compatible policyExpressions, presets, run creation, status, lifecycle control, the generic policy_wind_tunnel_bundle.v1 envelope, artifact, and evidence-pack availability. A weak bundle that only returns schemaVersion and runId is not conformant; it must expose the domain, pack, preset, branches, metrics, segments, timeline, decision gates, artifacts, and domain payload that the dashboard and outcome report can render without custom frontend code.

Install

Version 0.3.0 is prepared against the published shared contract release 1.2.0, including both native profile schemas. This validation tool is installed separately from the author kit; it is not a runtime dependency of domain services or simulation launchers. The remote-domain-conformance command is unchanged.

After SDK publication:

python -m pip install astraform-remote-domain-conformance==0.3.0

See the 0.3.0 release notes. Until then, build from the checkout using the published contract:

pip install -e './remote-domain-conformance-python[test]'

Run Against A Live Service

remote-domain-conformance \
  --base-url http://localhost:8092 \
  --domain-id my-domain

Require Population Builder catalog conformance:

remote-domain-conformance \
  --base-url http://localhost:8092 \
  --domain-id my-domain \
  --population-catalog

Validate a catalog file before the provider serves it:

remote-domain-conformance \
  --domain-id my-domain \
  --population-catalog-file ./population_catalog.json

Run Against A Local ASGI App

remote-domain-conformance \
  --app my_remote_domain.main:app \
  --domain-id my-domain

Baseline conformance performs the published v1 lifecycle once. To prove the stronger async worker transport/effect boundary, pass an explicit out-of-band profile:

remote-domain-conformance \
  --app my_remote_domain.main:app \
  --domain-id my-domain \
  --opportunity-worker-profile conformance/opportunity-worker-profile.json \
  --provider-artifact-digest "sha256:<64 lowercase hex characters>" \
  > build/conformance-run-report.json

Extract the nested opportunityWorkerConformanceReport, then sign that file in a separate protected job which runs no provider or repository code:

remote-domain-conformance \
  --sign-conformance-report build/opportunity-worker-conformance-report.json \
  --opportunity-worker-profile conformance/opportunity-worker-profile.json \
  --attestation-private-key /secure/release-ed25519-private-key.pem \
  --attestation-key-id partner-release \
  --attestation-output build/opportunity-worker-attestation.dsse.json

The protected signer recomputes the supplied profile digest and requires exactly one matching passing result for every classified tool. The signed conformanceReportDigest covers the report's passing baseline lifecycle, cross-attempt replay, and per-tool probe outcomes, plus its exact OCI subject binding. The public report schema is shipped in the opportunity-worker contract profile. Release automation must fail when the protected signer is unavailable; an ephemeral-key fallback is not a release proof.

The profile schema version is remote_domain_opportunity_worker_conformance_profile.v1; it contains exact toolEffects, one safe probe per tool, and optional personaConfiguration. It is not part of the remote-domain.v1 manifest schema.

Worker-profile responses are consumed from response.content, not decoded response text. Before a passing report can exist, the runner requires one exact JSON Content-Type, strict UTF-8, valid Unicode scalar text without U+0000, unique object keys, one JSON value, and lossless I-JSON/RFC 8785 numbers. Signed worker proof binds protocol responses to the manifest-advertised media type; the historical application/json fallback remains baseline-v1 compatibility only.

Native Profile Metadata

The native state-transform and reference-state schemas introduced in 0.2.0 remain included in 0.3.0, from published contract 1.2.0. The existing manifest validation accepts both capabilities.nativeSimulation profiles and validates complete customer-scoped read-only tool descriptors. Legacy manifests remain supported.

Validate a native metadata object with the existing validator:

from astraform.remote_domain.conformance.runner import RemoteDomainSchemaValidator

validator = RemoteDomainSchemaValidator()
validator.validate("manifest", provider_manifest)
validator.validate_native(
    "remote-domain.native-reference-state.v1",
    "stateMetadata",
    provider_checkpoint["data"]["nativeSimulation"],
)

The definition selects the exact canonical $defs entry: for example manifest, prepareHints, windowContext, stateMetadata or resultMetadata. The reference-state profile also supplies customerContext, readContext and checkpoint; the pure-state profile supplies readOnlyToolDescriptor and readOnlyToolPayload. Unknown profiles and definitions fail explicitly.

These checks validate wire shape and scope metadata. They do not prove native transaction atomicity, restart recovery, customer authorization, or complete native simulation behavior. The generic lifecycle CLI does not become native execution certification merely because it accepts a native manifest.

Run Policy Wind Tunnel Provider Conformance

remote-domain-conformance \
  --wind-tunnel \
  --base-url http://localhost:8092 \
  --pack-id my-domain-policy-pack \
  --preset-id conformance-proof

When the pack advertises opportunityGenerationFinalizationMode=PLATFORM_OWNED_V2, the wind-tunnel runner automatically executes a clock-driven V2 probe. It pins the immutable provider revision, cancels a generated run, finalizes it with a stable operation ID, validates the strict receipt schema, downloads and hashes every declared artifact, proves an exact replay, and rejects contradictory reuse of the operation ID.

V2 conformance requires a ScenarioLabFinalizationV2ProbeDriver backed by real OS processes. The driver supplies a started subprocess.Popen handle bound to the provider base URL under test, exposes the durable PREPARED receipt and artifact bytes, terminates the original provider, and launches a distinct process against the same durable state. The harness independently verifies that the old process exited and its endpoint is unavailable before accepting the replacement PID, then requires the replacement process to publish the byte-equivalent canonical /pack identity. Legacy V1 coverage is provisioned out of band as recovery state; the harness never launches a fresh PLATFORM_OWNED_V1 run. The driver also runs two distinct child processes against the configured receipt/artifact store so the harness can verify lock exclusion, cross-process visibility, and atomic publication. Both the successful and terminal-precedence paths cross a real process restart. Completion delivery is failed once deliberately and must retry independently without another FINALIZE request. The harness also injects each published artifact and receipt fault through the driver and verifies rejection through the provider HTTP boundary; an in-process ASGI app swap or a relabelled process is not restart proof.

Python Usage

import asyncio

from astraform.remote_domain.conformance.runner import ConformanceScenario
from astraform.remote_domain.conformance.runner import OpportunityWorkerConformanceProfile
from astraform.remote_domain.conformance.runner import PolicyWindTunnelConformanceScenario
from astraform.remote_domain.conformance.runner import ToolProbe
from astraform.remote_domain.conformance.runner import run_conformance
from astraform.remote_domain.conformance.runner import run_opportunity_worker_conformance
from astraform.remote_domain.conformance.runner import run_policy_wind_tunnel_conformance


report = asyncio.run(
    run_conformance(
        base_url="http://localhost:8092",
        scenario=ConformanceScenario(domain_id="my-domain"),
    )
)

print(report.to_dict())

worker_report = asyncio.run(
    run_opportunity_worker_conformance(
        base_url="http://localhost:8092",
        scenario=ConformanceScenario(domain_id="my-domain"),
        profile=OpportunityWorkerConformanceProfile(
            tool_effects={"lookup_case": "READ_ONLY"},
            tool_probes=(ToolProbe("lookup_case", {"caseId": "case-1"}),),
        ),
        provider_artifact_digest="sha256:<64 lowercase hex characters>",
    )
)

wind_tunnel_report = asyncio.run(
    run_policy_wind_tunnel_conformance(
        base_url="http://localhost:8092",
        scenario=PolicyWindTunnelConformanceScenario(
            pack_id="my-domain-policy-pack",
            preset_id="conformance-proof",
        ),
    )
)

print(wind_tunnel_report.to_dict())

Publishing Status

Release validation starts with the standalone package tests:

pytest remote-domain-author-kit-python/tests remote-domain-conformance-python/tests

PyPI artifacts are immutable. Do not rerun a publish for an already published version; run smoke-only verification or bump the SDK version.

Public package note: PyPI distributions expose this SDK implementation. Keep host runtime internals and domain-private logic out of this package.

HTTP safety limits

All provider calls stay on the configured HTTP(S) origin. Advertised operation paths must be relative paths; absolute URLs, authority paths (//host/path), backslashes, fragments and control/space characters are rejected before lifecycle operations start. Redirects are never followed. Localhost, private-network URLs and base URLs mounted under a path remain supported.

The Python runners accept limits=ConformanceHttpLimits(...) (exported from astraform.remote_domain.conformance). Defaults are 4 MiB per JSON response, 64 MiB per artifact/evidence-pack response, and a 30-second total deadline per HTTP request, including headers and streamed body. The same settings apply to polling, tool probes and restart requests. CLI equivalents are --max-json-bytes, --max-artifact-bytes and --request-timeout-seconds. Limits must be positive; the deadline must also be finite. The deadline is per request, not for a full run.

The runner requests Accept-Encoding: identity and rejects HTTP content compression before reading the body, preventing decompression from bypassing the byte limit. Providers must honor this request. ZIP artifacts are still supported as ordinary application/zip bytes. Both successful and error responses are bounded; exceeding a limit or hitting an HTTP transport timeout is a conformance failure, never successful crash or restart evidence. Connection-refused and disconnect errors remain available to the existing restart checks.

Imported app= implementations, custom transports and restart probe drivers run trusted Python in the harness process. In particular, HTTPX's ASGI transport buffers local app output before returning it; these limits do not sandbox local Python code or bound memory allocated inside that app. Use a separate provider process with base_url= to enforce the streamed network response boundary.