keble-tiktok

The source line requires keble-helpers>=1.54.0 and keble-data-infra-contract>=0.9.0 so FastMoss/EchoTik usage, failure, and Money evidence use the same public contracts in development and registry installs. Provider prices and durable retry policy remain outside this package.

Side effects if changes:

• Data Infra transports these usage events through the TikTok manifest.
• Changing the helper floor requires lock and raw API source-graph verification.

TikTok provider package for the Keble raw-data API. It owns FastMoss and EchoTik native clients, DTOs, channel-domain mappings, framework-free service classes, and explicit provider manifests.

FastMossServices and EchoTikServices accept data-infra contract requests and return normalized contract models. They do not construct FastAPI routers. keble-data-infra-api imports their manifests and owns all HTTP paths, exception handlers, settings, and lifecycle wiring.

Install the current public provider line with pip install "keble-tiktok>=0.8.0,<1". Version 0.8.0 requires the released data-infra contract and keble-db>=1.9.0; provider credentials remain runtime configuration and are never included in the distribution.

Release 0.8.0 is the REVIEW2 provider-recovery line. It binds typed FastMoss/EchoTik failure, usage, continuation, and cache-break evidence to Data Infra contract 0.9.0 by its immutable release commit; durable retries and provider pricing remain centralized in Data Infra.

Provider failure boundary

Each FastMoss or EchoTik public client call performs at most one physical HTTP request. The package never sleeps or retries internally: the centralized Data Infra runtime owns durable delay, retry budgets, circuit state, and replay. Native exceptions retain only typed and bounded evidence: HTTP status, reviewed business code, provider request ID, parsed Retry-After, and a sanitized message. Numeric and HTTP-date Retry-After values share the contract parser.

Provider manifests export FastMossFailureAdapter or EchoTikFailureAdapter. Known rate, quota, authentication, permission, invalid-request, invalid-response, transport, timeout, and availability failures map to the universal contract. The public manifest catch scope contains only the provider-owned error base and the package-owned local coordination error. A shared task supervisor translates HTTPX timeout/transport failures at the physical client boundary without retry, while a total Pydantic-core JSON decoder translates malformed syntax or schema into the provider's sanitized invalid-response type. Unexpected programming defects remain uncaught for worker supervision and Sentry. Unknown business codes and HTTP 402 remain unknown; this package never infers a payment problem without provider-documented billing evidence. The shared HTTP boundary marks physical request start inside a request-local metering scope. Every marked request emits one API_CALL usage event on success or failure; provider-specific preflight rejection and cache hits emit exact zero. A second physical mark inside one semantic invocation fails loudly as an undeclared hidden retry.

Reviewed FastMoss business codes remain semantically distinct: parameter codes 1/10001 require caller correction, 20001 is a missing provider resource, and transient 40001 requests a later retry. They must not collapse into one generic operator incident or imply a payment failure.

Side effects if changes:

• Data Infra HTTP responses and Platform durable recovery consume these exact manifest adapters and retry hints.
• Moving retry into a provider client would multiply physical calls, usage, cost, and attempt counts behind the central scheduler's back.
• Broadening payment classification would create false operator billing gates.
• Registering framework exceptions would make HTTPX/Pydantic upgrades part of the cross-repository scheduling contract and hide local programming defects.

Provider-native detail models stay inside this package. Public results expose only normalized fields; shared monetary values use keble_helpers.Money. Production docstrings follow the split keble_data_infra_contract plus concrete FastMossServices/EchoTikServices composition; retired monolith and ApplicationService terminology remains only in explicitly historical records.

FastMoss price and GMV fields are provider major-unit decimals. The mapper validates that each value is finite, non-negative, and exactly representable, then delegates scaling to Currency.minor_unit_exponent and Money.from_major_units. It does not maintain a TikTok-local zero-decimal list and does not call the cents compatibility API. Consequently CLP/JPY use exponent 0, IDR/HUF/TWD use exponent 2, and BHD uses exponent 3 on the same canonical wire contract.

Side effects if changes:

• CommerceListing prices and GMV evidence consumed by Data Infra and Data Platform depend on this exact major-to-minor conversion.
• Rounding or provider-local exponent tables would silently distort ranking, forecasting, and historical financial facts.
• Values with excess precision remain omitted rather than guessed or rounded.

Distinct trending resources

Version 0.6 exposes native TRENDING operations rather than adapting search:

• FastMoss: products and creators through its dated product/creator boards;
• EchoTik: products, creators, sellers, and channel-wide videos through four distinct ranking endpoints and schemas.

The shared adapter maps lookbackDays to the latest completed day, ISO week, or month. Provider cursors are page-local and remain hidden behind Data Infra's opaque central session. Bare-list EchoTik responses are definitively exhausted only after an empty page (or a documented reachable product-page cap); a sparse non-empty page remains UNKNOWN, so filtering or dedupe cannot create false source exhaustion.

Product ranking applies the same contract at every adapter hop: TikTokProductRankingResult.total is None when EchoTik reports no global board count, while has_next independently carries definitive continuation evidence. TikTokCommerceProvider consults a numeric total only when a source actually supplies one, so neither a full nor sparse EchoTik page can become false exhaustion from its page-local item count. FastMoss continues to publish and use its real board total.

A successful rank envelope whose selected sort metric is absent is a malformed provider response, not an invalid caller request. Product, creator, seller, and video sources therefore emit EchoTikInvalidResponse for those rows while keeping unsupported sort choices and invalid pagination as EchoTikInvalidRequest. Data Infra can consequently apply provider-response recovery without quarantining a valid query as user error.

Every item retains its genuine resource model and a neutral summary plus typed metric/board-position evidence explaining why it stands out. Products use CommerceListing, creators use CommerceCreator, sellers use CommerceStore, and channel-wide videos use ShortVideo. The video board has its own typed /api/v3/echotik/video/ranklist DTO/source and is not the product-linked video list. Its embedded product references are parsed with typed Pydantic Json; transient playback URLs may cross the raw response but are never stable identity and the platform does not persist them.

Provider packages emit all source occurrences and usage events. Query-selected entity/family dedupe, buffered pagination, and fixed-time session invalidation remain owned by the central Data Infra coordinator.

uv lock --check
uv run pytest -q -m "not live and not slow and not eval and not local_stack and not db_stack and not container"
uv run npx --yes pyright .
uv build

HTTP transports and environment-proxy behavior are constructor-injected. Direct clients default to trust_env=False for deterministic library tests; the service builders explicitly enable production environment proxies. The runtime dependency includes HTTPX's SOCKS transport so a configured SOCKS4/5 proxy is supported rather than failing during client construction.

Durable provider cache and batches

The API may inject one ProviderOperationCache into each service builder. The cache stores validated contract JSON in an API-owned Mongo collection keyed by provider, operation, contract version, and the hash of the complete canonical request. Empty successful responses use a shorter TTL. Redis owner-token leases ensure concurrent identical misses call upstream once, then all contenders recheck Mongo before provider I/O.

The current normalized response-cache identity is v2. This is an intentional clean contract break: current reads never deserialize legacy v1 payloads, while the existing TTL index retains and then expires those rows without a broad destructive delete. A later schema change must bump the package-owned constant and prove malformed legacy rows remain unreachable.

ProviderResponseCacheRepository.ensure_indexes() must run during API startup. It creates the exact unique compound read index plus expires_at TTL index; drop_obsolete_indexes() removes indexes whose query shapes no longer exist.

FastMoss search and EchoTik video services expose service-only aload_many_* methods. They deduplicate exact requests, bound concurrency where no native batch exists, drain every sibling to terminal state, and reconstruct the caller's original order. Paid batches require the per-item durable cache and are rejected before I/O when it is absent. Therefore one rate-limited item cannot return early, abandon a successful sibling cache write, and force that item to be purchased again on retry. No HTTP batch route is generated.

When the API initializes Sentry, exact cache reads, Redis lease acquisition, real provider work, and service-only batches emit nested spans. Span data is limited to stable provider/operation/outcome/count metadata; request keywords, response payloads, and credentials are never attached.

Each service builder also accepts the package-neutral UsageAccountingRecorderProtocol. One real FastMoss/EchoTik upstream interaction emits one API_CALL event with provider, operation, and duration whether the physical request succeeds or fails. The recorder is deliberately invoked inside cache-miss handlers, so cache hits never replay old billable usage. Prices and currencies remain owned by the consuming platform.

Real cache tests require isolated Mongo/Redis and use:

KEBLE_BACKEND_ENV_FILE=/path/to/keble.backend/.env \
RUN_INTEGRATION=1 RUN_REAL_DB=1 RUN_LOCAL_STACK=1 RUN_DB_STACK=1 \
  uv run pytest -q -m "not live and not container"