keble-data-infra-contract

Install the provider-neutral public contract with pip install "keble-data-infra-contract>=0.10.0,<1".

Framework-free, provider-neutral raw-commerce contracts. This distribution owns validated request/response models, provider protocols, operation keys, manifests, and typed domain errors. It intentionally has no FastAPI, database, HTTP client, provider, platform, or idea dependency.

The distribution declares only registry-resolvable runtime dependencies. Do not add workspace-relative tool.uv.sources here: downstream Git/subdirectory installs must build this contract without inheriting paths outside the repository checkout.

Provider packages own implementations and native DTOs. The raw API consumes provider manifests and constructs HTTP paths from their typed operation keys.

Page[T] and RankingSnapshot[T] carry a provider-neutral usage list of keble-helpers.UsageAccountingEvent. Providers measure physical tokens, requests, items, media, or bytes; this contract only transports those facts. It must never contain prices, currency conversion, allocation policy, or deployment secrets. Cache hits return an empty list so old upstream spend is not replayed to consumers.

Version 0.2.0 is the exact-usage release. It requires keble-helpers>=1.58.0 and preserves media/storage/compute quantities plus fully failed video-operation attempts without reconstructing prices. The distribution has no Git or workspace source override, so consumers resolve the same public contract in clean build and runtime environments.

Version 0.3.0 adds first-class evidence-bearing TRENDING operations without aliasing search or rank. Products, creators, sellers, and videos have distinct typed query contracts; every result carries stable provider identity, optional product-family identity, structured standout evidence, source exhaustion, and shrink diagnostics. Providers report raw source truth and never apply cross-page dedupe. The central raw API alone applies the request's none, entity, or product_family policy through an opaque continuation session.

Version 0.4.0 adds the provider-neutral indexed-commerce boundary required by Shopify without reusing TikTok board or Amazon ASIN contracts:

• StoreSearchQuery, StoreIdentityLookupQuery, StoreProductsQuery, and ProductIdentityLookupQuery;
• optional product-search cursors that cannot be combined with numbered pages;
• PageInfo.end_cursor while preserving numbered-provider metadata;
• CommerceStoreObservation with typed popularity values and stable identities;
• provider-neutral CommerceCatalogCoverage with immutable full/window scope, terminal completeness, absolute cutoff, exhaustion, and an exact one-outcome- per-discovered-product accounting invariant;
• Store Leads product and image counts retained as separate sizing signals;
• indexed query provenance separated from ordered Store Leads/storefront source provenance and immutable artifact ids.

Provider-specific lifecycle values are normalized into closed universal store and listing state enums. Unknown fields, untyped extension bags, inferred identities, and source-provider collapse remain forbidden.

request = ProductTrendingQuery(
    query="red clothing",
    market="US",
    dedupe_mode=TrendDedupeMode.PRODUCT_FAMILY,
)

Blank product intent is valid category browsing. A filtered/dedupe-empty page may remain continuable; only definitive upstream evidence can set TrendExhaustion.EXHAUSTED. TrendEvidence.summary is display-ready but must also carry an auditable metric, event time, or provider-board position.

Version 0.5.0 replaces string-only upstream errors with a single immutable DataInfraFailure payload and DataInfraFailureError. Provider manifests now require a ProviderFailureAdapterProtocol; the raw API registers the declared native exception classes without importing Keepa, Amazon, TikTok, FastMoss, or EchoTik error types. Retry/payment/operator combinations are validated, payment requires reviewed evidence, and physical usage/capacity observations remain exact and sanitized. The minimum helper line is 1.52.2, which preserves negative provider token debt and a separate exact refill-reduction rate instead of rejecting, clamping, or misclassifying those capacity facts. parse_retry_after(...) is the single shared parser for positive delta-seconds and canonical IMF-fixdate headers. It ignores malformed, past, zero, and overflowing hints, returns timezone-aware UTC, and does not apply Platform's versioned maximum-hint policy.

IdempotencyKeyConflictError is the provider-neutral signal for a caller key already bound to another operation/request fingerprint. Its safe message never contains the caller key. Data Infra API owns Redis hashing, claim/replay, HTTP 409, and retention; the contract deliberately owns no datastore policy.

Version 0.6.0 makes the expected non-success HTTP body unambiguous through DataInfraFailureEnvelope. The envelope contains exactly one validated failure and rejects parallel detail or extension bags; Data Infra API and Data Platform must therefore evolve and release their serializers/parsers in the same wave. A source-aware CI guard compares the current contract src/ tree with the newest package-scoped annotated tag and rejects changed source whose package version was not advanced. Package metadata regressions also fail closed.

Version 0.7.0 carries the same canonical DataInfraFailure inside failed video operation outcomes. Job-attempt and Result projections therefore retain the producer-owned disposition, operator action, retry time, sanitized native identifiers, usage completeness, and exact usage events. The normalized operation and nested failure must contain the same usage list; divergent copies fail validation rather than creating two financial truths.

Version 0.8.0 extends the video-ingestion repository protocol with one bounded list_stalled(now, limit) read. Implementations return only nonterminal rows whose owner is absent or expired; recovery callers must pass each returned completion event back through the existing atomic claim method. No reset, force-unlock, provider response, price, or database shape enters the framework-free contract.

Version 0.8.1 makes failure transport self-identifying and centralizes local contract-error conversion. Every canonical non-success response advertises X-Keble-Data-Infra-Failure-Schema: 1 beside the existing DataInfraFailureEnvelope. Consumers must require both this exact header and a valid envelope; legacy detail, malformed JSON, unknown schema versions, and wrong media types are untrusted responses rather than guessed failures. failure_from_contract_error(...) is the one framework-free conversion for operation/provider/request/idempotency rejections and can retain exact usage captured after provider I/O. ProviderRequestValidationError.after_provider_io distinguishes those post-I/O validations from preflight known-zero rejection.

Version 0.9.0 makes catalog coverage financially exact: every discovered product must resolve to exactly one terminal processed, unchanged, skipped, or failed outcome. Partial windows remain representable, but an omitted or double-counted product can no longer pass validation and silently distort media, provider-unit, or per-product forecasts. This is a first-release contract step; pre-release invalid projections use the guarded clean break rather than a second permissive wire schema.

Version 0.10.0 prepares the accepted intelligence source-contract surface for publication. It retains the provider-neutral capability, evidence, identity, relationship, revocation, and exact-route admission contracts as one releaseable package. SourceOperationAdmissionProjection exposes only selected stable cost profile keys, one typed positive finite attempt bound per key, the exact quota selector, and positive workload quantities that exclude concurrency leases. Its schema revision and fingerprint are deterministic. Data Infra policy/catalog remains authoritative for keys, bounds, quantities, rates, money, endpoints, secrets, recovery policy, and live quota values. The release gate verifies the package-scoped version bump and imports every contract symbol used by the accepted G2 Platform source from an isolated built artifact; it does not bump, tag, publish, or otherwise release the composed API.

Side effects if changes:

• Data Infra API must attach the header to every expected non-success response.
• Data Platform must use one total versioned parser and sanitize untrusted bodies.
• Invocation scopes must retain captured usage before converting post-I/O errors.
• Provider packages still own native failures; this factory accepts only contract errors and does not classify provider-native exceptions.

Side effects if changes:

• Data Infra API and repository test doubles must implement the same stalled eligibility boundary.
• Active claims are authoritative and cannot be stolen by an operator scan.
• Adding another nonterminal ingestion status requires a coordinated protocol, index, implementation, and recovery-test update.

The minimum helper line is 1.58.0. Currency amounts retain the stable {"amount":{"value":...},"currency":...} wire shape, while amount.value now has one explicit meaning: ISO minor units at Currency.minor_unit_exponent. Use Money.from_minor_units for provider integer units and Money.from_major_units(Decimal(...)) for human/provider major-unit decimals; do not reintroduce cents or fixed-two-decimal conversion.

Side effects if changes:

• provider packages must adapt every declared native exception to this model;
• Data Infra API owns HTTP/Retry-After serialization of the payload;
• Idea/Platform own their separate durable job/recovery persistence models.
• provider adapters must use parse_retry_after rather than redeclaring HTTP date or numeric parsing.

Side effects if changes:

• provider packages and the raw API serialize these exact camel-case events;
• the Ideas platform deduplicates producer attempt keys and prices quantities with its own immutable profile effective at the event time.

uv run --package keble-data-infra-contract pytest -q keble-data-infra-contract/tests
uv run npx --yes pyright .
uv run python keble-data-infra-contract/src/keble_data_infra_contract/release_guard.py
uv build --package keble-data-infra-contract

Contract values inherit ContractModel, which accepts snake-case Python names and camel-case wire aliases, rejects undeclared fields, and is frozen. Persisted Mongo shapes do not belong here; provider/API packages own SchemaBase / MongoObjectBase aggregates when durability is required. ContractModel and the deliberately producer-tolerant Job/Result projection both configure Pydantic through keble-helpers.PydanticModelConfig; a local ConfigDict convention would split alias and validation ownership again.

Money, Currency, and AmazonMarketplace are imported from keble-helpers. Provider-native response details remain inside the provider package instead of crossing this boundary through an untyped extension bag.

Video enrichment uses the same framework-free boundary: immutable asset, submission, job, completion-event, result, and normalized-projection models plus one VideoEnrichmentGateway Protocol. HTTP submission and Mongo correlation remain implementations owned by keble-data-infra-api.

Each video operation outcome carries canonical UsageAccountingEvent rows and an opaque producer price-profile identity. The normalized projection preserves both the per-operation status (including partial failure) and a flattened usage view. It deliberately omits producer estimates and monetary values: ASR/OCR route policy stays in Video Enrichment, while the Ideas platform resolves its own immutable rates.

The job read also retains analysisOperationAttempts for every whole-workflow retry. This is the accounting path for fully failed jobs and for failed provider calls that precede a later successful result. Consumers must deduplicate by the producer attempt key carried by each usage event; they must not infer zero cost from the absence of a result.

Authorized provider transcripts use one canonical VideoTranscript/ VideoTranscriptSegment shape across submission, authoritative result reads, and the local projection. Reuse evidence binds the normalized transcript checksum to the data-infra source reference and media SHA-256; a title/description such as ShortVideo.caption is not a spoken transcript and cannot bypass ASR.

Portable commerce metadata

CommerceProduct may carry provider-neutral description, feature bullets, product type, full category path, source-language hint, and a precision-aware release-date interval. RECENTLY_RELEASED is the canonical trend reason; RECENTLY_LISTED is accepted only as a deprecated input alias and never emitted. Consumers must not infer absent metadata or replace release time with offer-history time.

Listing-authoritative variant candidates remain bounded assertions over listings embedded in the same CommerceProduct. Every VARIANT_OF assertion must include the candidate's own listing identity, but it need not include the response-authoritative listing: a child-ASIN response may therefore retain an observed sibling-to-parent representative assertion. External endpoints and assertions filed under a candidate they do not mention are rejected.

Shopify listing authority v2

One normalized Shopify product page is one listing-authoritative CommerceProduct. Its source_identity and only embedded CommerceListing use PRODUCT_LISTING with the normalized keble_shopify/shopify_product_key; the Product GID remains channel_product_id plus a store-scoped shopify_storefront/shopify_product_id alias. Product handles and page URLs remain mutable locator/display evidence and are not canonical identities.

Sellable variants remain bounded CommerceProductVariant children. Each child uses PRODUCT_VARIANT with keble_shopify/shopify_variant_key and retains its store-scoped shopify_storefront/shopify_variant_id alias. Neither the Product GID provider nor the former variant-key listing namespace can create a second canonical Product/ProductListing/ProductVariant identity.

Every normalized product aggregate contains exactly one occurrence-and-checksum bound PRODUCT_LISTING_SOLD_BY_STOREFRONT assertion. The registered mapping now uses shopify-product-projection-v2 and keble-shopify-index-v2; provider/API cache revisions must treat v1 per-variant listing and relationship identities as incompatible rather than adapt or alias them.

Side effects if changes:

• keble-data-infra-api.integrations.shopify.mapper must emit one listing and one SOLD_BY assertion per product page.
• Shopify provider projections must retain normalized product/variant/store keys and original Product/Variant GIDs separately.
• Platform publication, store-expansion metrics, and relationship ingestion count product pages/listings independently from their variant children.

Source intelligence evidence

Every provider operation carries a SourceOperationCapability validated against its registered Pydantic request model. ProviderManifest derives a stable semantic revision and a provider-affine SourceOperationSelection; runtime health and catalog fingerprints are separate values.

SourceOperationCapabilitySnapshot is the one secret-free, exact-RouteKey projection for Campaign forecast/revalidation. It retains the registered request/result schema revisions, manifest capability revision, exact selection and typed capability, fingerprints, and one exact bounded admission projection. Its freshUntil cannot exceed 30 seconds after healthObservedAt; UNKNOWN health remains an admission failure, not an implicit provider probe or availability claim.

MetricObservation now supports precise measured/interval time, raw source period labels, coverage, confidence, and strict value/unit pairing. SourceRelationshipAssertion binds provider-namespaced subject/object IDs to a reviewed mapping, relation metrics, time, and provenance without accepting a canonical edge. Exact-byte SourceArtifactRef and trusted SourceShareabilityEvidence compose SourceResultEvidence at the provider durability boundary. Evidence sourceService is the literal data_infra on both direct Python construction and camel-case JSON validation, so provider payloads and cache documents cannot impersonate another evidence authority. The open SourceServiceCode remains limited to non-evidence revocation and authority transport. Public pages, rankings, trending results, and failure envelopes carry bounded exact-response evidence; item and relationship associations distinguish direct normalization bytes from selection/traversal bytes. A provider cannot publish a result until its parser input is durably captured, checksum-bound, and authorized for the current consumption.