Keble Scraper Contract

keble-scraper-contract is the framework-free request, lifecycle, artifact, proxy, callback, and operator identity boundary for Keble's asynchronous scraper. Generic callers can submit only GET/HEAD public-resource requests. The sole POST exception is the registered, typed Shopify Storefront GraphQL profile; callers cannot supply GraphQL text, mutations, arbitrary headers, or credentials.

All values are immutable Pydantic v2 models, reject undeclared fields, serialize with camel-case aliases, and expose bounded enums instead of arbitrary TTLs or transport flags.

ScrapeArtifactPin is the ordered retention-policy command for one stable artifact/owner identity. Its required positive retention_revision makes finite archive expiry, indefinite latest retention, replay, and out-of-order delivery distinguishable. A producer must increment that revision whenever it changes expires_at; reusing one revision for a different expiry is a contract conflict.

Version 0.3.0 releases that ordered retention contract together with the complete typed failure boundary. The new pin field is intentionally mandatory: an older producer cannot silently recreate indefinite paid-storage retention, and a delayed delivery cannot reverse a newer finite/indefinite policy. Shopify, Scraper API, and Data Infra therefore repin in the same release wave.

Version 0.2.0 replaces free-string attempt/job/callback failures with one discriminated contract:

• ScrapeLocalFailure owns caller-auth, request-policy, runtime, capacity, and operator failures through a finite code/origin vocabulary;
• ScrapeUpstreamFailure owns target-resource, paid-egress, and browser-provider failures with universal kind/disposition/action, exact capacity evidence, bounded native status/request identity, and optional retry time;
• supplier payment is valid only for a paid supplier origin with reviewed documented-code or account-status evidence. A target 402, a rate limit, quarantine, or missing credential never implies payment;
• attempts, terminal jobs, signed completion events, and HTTP errors use the same typed failure family. Success forbids failure evidence and every failed terminal state requires it.

The public wire uses ScrapeFailureEnvelope for non-success HTTP responses. The contract source/version guard compares src/ with the latest package-scoped annotated tag and rejects source drift under an existing version or metadata regression.

Side effects if changes:

• scraper API, Shopify polling/callbacks, and frontend mappings must release in the same wave;
• artifact-pin revisions and Shopify pin-outbox payloads must remain one coordinated wire contract;
• Data Infra may adapt upstream failures but must not reinterpret local failures as provider payment.

uv run --package keble-scraper-contract pytest -q keble-scraper-contract/tests
npx --yes pyright .
uv run python keble-scraper-contract/src/keble_scraper_contract/release_guard.py
uv build --package keble-scraper-contract