Skip to main content

Media ingestion, processing and analysis service for keble short videos (PlayRef -> EnrichmentResult).

Project description

keble-video-enrichment

短视频媒体处理与理解服务。可通过 keble-data-infra-contract 接收通用视频引用 (ShortVideo + PlayRef), 完成安全摄取、校验、checksum、ffprobe、音频/关键帧提取、ASR、OCR,输出结构化 EnrichmentResult 供 Data Infra / Search / GraphRAG 消费。

keble-tiktok 负责找到视频,keble-video-enrichment 负责处理和理解视频。

0.4 shared-catalog production runtime

The supported mass-volume entrypoint (Dockerfile.paddle) compiles four typed Infisical/local environment documents before it resolves a credential or opens provider/storage clients:

  • LLM_CATALOG: logical ASR/vision/local models, physical endpoint bindings, per-binding quota evidence, token prices, and credential references;
  • SERVICE_COST_CATALOG: hosted-audio duration, local worker compute, and OSS prices with effective windows and immutable fingerprints;
  • SERVICE_QUOTA_CATALOG: non-LLM provider limits such as OSS requests;
  • VIDEO_ENRICHMENT_POLICY: explicit primary/alternate model keys, Paddle-first OCR fallback, five-frame midpoint sampling, timeout, zero SDK retries, strictness, and per-video budget.

Credential values remain separate environment variables named by the catalog. For one logical model with multiple healthy endpoint bindings, the runtime randomizes among quota-admissible endpoints before physical I/O. It may skip an endpoint only before I/O when admission is delayed. After any physical attempt, the durable whole-Job retry owns the next call; there is no invisible SDK retry or same-attempt provider fallback. /ready exposes only the four secret-free catalog/policy fingerprints so operators can detect replica drift.

The local Paddle and Whisper bindings use LOCAL_RUNTIME plus local:// identities and no credential. Their non-zero cost is measured worker compute duration under SERVICE_COST_CATALOG; hosted ASR is billed by measured audio duration and vision fallback by returned token usage. Estimation and execution share exactly five equal midpoint bins, so forecast frame counts cannot diverge from the physical extractor.

The base Dockerfile and its scalar parser remain a clearly labelled compatibility image for old canaries only. It is not selected by either production deploy repository. New production configuration must not set the old OPENAI_*_MODEL, AZURE_*_DEPLOYMENT, KVE_ANALYSIS_* routing/cost, or compiled-quota-file variables; .env.example documents only the catalog path.

Side effects if changes:

  • keble-data-infra-api observes catalog fingerprints and exact usage on every completion attempt;
  • Data Platform finance and forecasting depend on immutable selected price facts rather than current environment values;
  • keble-data-infra-deploy pins Paddle for ordinary production and may pin Whisper only as an explicit offline engineering canary.

Review2 retry and failed-usage release line

The next public package resolves keble-helpers>=1.57.0, keble-db>=1.9.0, and keble-data-infra-contract==0.9.0 entirely from the registry. No Git source override or repository credential is required to install or build the service. Existing 0.1.x deployments have no production data migration obligation; the first operational rollout may start from a clean database and object prefix.

Release 0.3.1 extends the provider-call crash-window fix through bounded history recovery and the guarded first-release clean break. Before ASR or OCR can start, the Job aggregate commits a persistence-only operation-start marker. Each independently completed branch immediately replaces its marker with exact terminal usage; timeout, process restart, or an unexpected task defect converts any surviving marker into an explicit billing-unknown attempt before completion or retry. Public Job schema remains v2 and exposes only terminal attempts, never the internal in-flight markers. Expected provider failures become typed OperationFailure results; arbitrary code defects reach structured logs and optional Sentry supervision instead of being mislabeled as provider outages.

Hosted ASR, OCR, Azure Speech, and media HTTP clients are now application- lifespan resources. Concurrent videos reuse their HTTP/TLS pools, and shutdown drains admitted Jobs before supervising every client close. Azure Speech Batch deletes every submitted job on success, provider failure, or polling timeout; cleanup rejection is observable but cannot replace the paid transcription outcome. Optional OCR fallback preserves successful primary observations and usage when its typed provider call fails, while unexpected fallback defects still propagate.

GET /v1/enrichment/jobs/{job_id} now transports immutable analysis_operation_attempts, including usage from failed ASR/OCR calls. A retryable failed idempotency key revives the same stable Job through its next whole-job attempt; completion-event identity includes that attempt ordinal, and Mongo startup removes the obsolete unique event.job_id index. Replaying the same terminal attempt remains idempotent while a later attempt can publish its own financially relevant completion.

Side effects if changes:

  • Data Infra consumes terminal attempt history and canonical usage from the 0.9.0 contract before the Ideas platform writes immutable cost and recovery facts.
  • ASR/OCR route selection, quota profiles, price evidence, and credentials stay environment-owned by this service; package publication does not activate a route or claim a 100K/day production quota.

REVIEW2 bounded lifecycle checkpoint (release pending)

The current fix/review2-video-usage-recovery topic adds two measured capacity boundaries without changing the public Job v2 response:

  • Every production OSS upload first reserves its exact object key in Mongo with a storage-revision fence. Cleanup claims only indexed due lifecycle rows in a configurable batch; cold reads use the unique artifact identity. The service no longer calls OSS ListObjects or HEADs every live object during cleanup.
  • A terminal Job carries an internal completion_pending marker. Startup reads at most KVE_COMPLETION_OUTBOX_BATCH_SIZE indexed pending Jobs, stages the immutable outbox event, starts Result retention, and only then clears the Job marker. Historical terminal Jobs are never scanned on startup.
  • Job/Result TTL starts at completion_staged_at; published outbox TTL starts at published_at; deleted artifact tombstone TTL starts at deleted_at. Active, pending, unpublished, and cleanup-leased rows have no eligible TTL timestamp.

The configuration is environment-owned so local .env and production Infisical use the same typed contract. See 2026-07-19-review2-video-capacity.md for the reproduced full-scan baselines, 10K query plans, and 100K corpus proof. This checkpoint is not a release or main-merge claim; the coordinated REVIEW2 release train owns those final actions.

WP8 includes scripts/reset_pre_release_history.py for the approved first-release namespace break. It is DRY_RUN by default and permanently disables APPLY outside local/test environments. APPLY requires the exact Mongo database, stopped-writer confirmation, Mongo rollback reference, artifact-namespace backup/reset reference, stable campaign identity, and a new immutable audit path. It deletes only the four explicitly owned Job/Result/completion-outbox/artifact-lifecycle collections, never discovers or drops a database, collection, bucket, or adjacent namespace. The full operator sequence and rollback condition live in docs/operations/runbook.md.

财务用量与批量准入

本服务只拥有媒体执行事实与路由策略,不拥有货币价格。每个 ASR、OCR、本地媒体处理结果均发出 UsageAccountingEvent,精确区分音频秒、图像帧、token 与本地 compute millisecond;事件携带不可变的 producer attempt/profile identity,但不携带单价或金额。终态 Job 以 schema v2 保存每次分析尝试,因此 全失败调用与“先失败、后重试成功”的调用都能由 Data Infra 传给 Platform 财务账本,不能因没有 Result 而消失。修改这些事件或 identity 时,必须同时检查 application/analysis.pyapplication/service.pydomain/job.py 与 Data Infra consumer contract。

ASR 与 OCR 在完成共同的媒体准备后并发执行,并先分别预留预算;最终账面事实按实际 provider 响应结算, 不会把预留值冒充实际金额。批量准入由可替换的 QuotaAdmissionStore 管理:测试可用进程内实现,生产必须 使用 Redis Lua 原子脚本,在所有 replica 之间共享并发 lease 与 UTC 日额度。默认上限为 16 个并发 Job、 每天 7,500 个 Job,lease 必须长于 Job timeout;这些是安全容量设置,不是 provider RPM/TPM 或价格。 生产如缺 Redis、额度小于目标视频量或 lease 配置不安全会在启动时 fail closed。

Hosted ASR/OCR 另有一层精确到 logical model / endpoint / deployment / region / billing-mode 的 immutable quota profile。 生产直接从 LLM_CATALOG 的 reviewed binding quota evidence 编译这些 profiles;旧的 scripts/compile_provider_quota_profiles.pyKVE_PROVIDER_QUOTA_PROFILE_PATH 仅保留给 compatibility canary, 不得用于 Paddle/Whisper catalog entrypoints。 Redis Lua 会原子预留 RPM、input/output/total TPM、并发、audio-seconds/day 与 provider credits/day;拒绝不会 留下部分计数,crash lease 会按 TTL 回收。默认 5,000 个三分钟视频、30% headroom 的 preflight 至少要求 ASR 5 RPM、remote OCR 55 RPM,因此当前 3-RPM Whisper deployment 会明确拒绝启动,而不是在运行中排队到 第二天。VIDEO_ENRICHMENT_POLICY.sdkRetries=0 是 production 要求:physical retry 必须成为可计量的 Job attempt,不能藏在 SDK 内。旧 profile 示例见 docs/examples/provider-quota-profiles.example.uncompiled.json, 容量与 live 证据见 docs/benchmarks/2026-07-13-video-capacity-and-live-calibration.md

项目是什么 / 不是什么

:由 Keble 根据现有系统边界实现的媒体摄取与分析服务;包含 Job 状态、幂等、重试、部分成功、 Artifact 生命周期和 ASR/OCR 规范化。AWS Media2Cloud 仅作为成熟架构参照,用于验证职责分离方向, 不是实现模板或运行时依赖。

不是:视频数据源(不调用 EchoTik/FastMoss 私有 API);通用 Contract 仓库(不定义 ShortVideo); Web 管理后台/MAM/内容审核平台/视频编辑工具。

为什么独立

媒体处理是长耗时、重依赖(ffmpeg/ASR/OCR)、易部分失败的工作负载,与轻量 contract 层和渠道 连接器的定位冲突。keble-data-infra 执行计划的决策 D5 已锁定独立仓库。 详见 ADR-001

与相邻仓库的关系

  • keble-data-infra-contract:本项目 canonical provider-failure 与媒体引用的唯一上游 contract package(DataInfraFailure/ShortVideo/PlayRef/ProviderId);当前 Review2 release line 固定使用 registry 发布的 0.9.0 contract。Failed operation outcomes use this contract to retain provider-owned retry/payment decisions and exact usage completeness. 所有 image 都安装该轻量 contract,并只从 registry 解析运行依赖,因此 build 和 runtime 均不需要 repository credential、Git client 或源码 checkout。
  • keble-data-infra-api:通过 HTTP 提交/读取 enrichment,并实现 contract 中的 consumer Protocol;本项目不导入 API/runtime distribution。
  • keble-tiktok:互不依赖。它产出 PlayRef,本项目消费标准化后的引用;渠道差异在进入本项目前 已被抹平。详见 ADR-004

成熟参照与 Keble 自主决策

AWS Media2Cloud 用于验证稳定资产身份、摄取与分析分离、完整性校验、持久化任务状态和完成通知等原则。 本项目没有复制其 Step Functions、Lambda、DynamoDB、SNS、MediaConvert、Transcribe、Rekognition 或完整 部署模板。Keble 根据轻量部署、现有 MongoDB/Redis、可替换 AI Engine、合规保留策略和 Data Infra 边界, 独立定义并实现了 Domain Contract、Application Workflow、Repository、Adapter、部分成功和 durable CompletionEvent 交付。

代码与测试证据见架构文档;逐组件参照范围和未采用 项见 Media2Cloud mapping

MVP

一个 PlayRef → 安全获取 → 校验 + checksum → ffprobe → 音频/关键帧提取 → 一次 ASR + 一次 OCR → 结构化 JSON → 清理临时原始媒体。合规 gate 通过前,开发全程使用本地合成测试视频。

目标架构

当前实现通过 Port 连接 FastAPI、异步本地 Runner、MongoDB、Aliyun OSS 和 Redis Stream;这些选择来自 Keble 的部署条件与已批准决策,不表示已具备 AWS 式多机编排或生产弹性。 全部图与规则:docs/architecture/keble-video-enrichment-architecture.md

当前状态

Phase 00(规划)与 Reference Baseline Reconciliation 已完成;Phase 01(Project Baseline and Contracts)、Phase 02(Single Video Ingestion)、Phase 03(Media Processing)与 Phase 04(ASR/OCR) 已完成;Phase 04 的 Azure OCR adapter 与独立 REST service 已使用仓库自有合成媒体通过真实 gpt-5.6-luna smoke;Azure hosted Whisper 与 Azure Speech Batch 已在同一组自有 EN/ZH fixture 上完成 真实对比,production route 仍等待 owner threshold/label approval。Phase 05 已完成 MongoDB 编排持久化; Phase 06 已完成 REST Job/Result 查询、 durable Mongo outbox、keble-db Redis Stream publisher 与 v1 consumer contract。真实 Mongo/Redis/ provider 测试保留显式 gate。 Phase 07 应用开发已完成:通用 Artifact Store、Aliyun OSS adapter、配置/wiring、结构化日志、Prometheus、 quota、清理调度、Linux x86_64 Docker runtime、runbook 与安全评审均已落地。新的深圳开发 bucket 已通过 delete/list preflight、put/head、完整与 range get、checksum、最终 delete 和空前缀审计。Bucket-level 历史 live gate 已验证 oss:ListObjects 与 object-level CRUD policy;当前 runtime 的冷启动 lookup 与 TTL cleanup 已不再调用 ListObjects。它们分别使用 Mongo unique point read 与 indexed due-work claim,仅对真正 到期的 exact object 执行 metadata/delete;users/phase-07-live-e2e/ 下残留对象计数为零。 EnrichmentSettings.from_env() 只拥有 storage/job/media mechanics; VideoCatalogEnvironmentSettings.from_env().compile() 是 production model、route、price 与 quota 的单一强类型入口。 显式构造 adapter settings 时直接传 Pydantic 字段,不再把 _env_file 这个框架内部参数散落到调用方。 所有 settings loader 都把 .env.example 中空白的可选成本/凭证值视为“未配置”,不会把空字符串 误解析成整数,也不会把未知成本错误地当作零成本;仍为必填的 benchmark threshold 保持 fail-closed。 示例凭证保持空白而不是可被误认作真实 key 的占位字符串,因此复制 .env.example 不会抢先选择 错误的 OpenAI/Azure provider;真实 secret 只写入 gitignored .env 或部署 secret store。 共享 dotenv 同时包含 OPENAI_*_MODEL 与 Azure deployment 时,model/provenance 也严格按已选 provider 解析,OpenAI 默认值不得覆盖 Azure deployment。若音频与 OCR 位于不同 Azure resources, AZURE_KEY_ASR/AZURE_ENDPOINT_ASR 独立拥有 ASR credential boundary,而 AZURE_KEY/AZURE_ENDPOINT 只归 OCR 使用;ASR 不会静默 fallback 到 OCR resource,禁止把一个 resource 的 key 与另一个 endpoint 拼接。 Azure OCR 使用 Responses API,AZURE_OCR_VERSION 必须为 2025-03-01-preview 或更新版本; 兼容 OpenAI Audio route 独立使用 AZURE_ASR_VERSION/AZURE_ASR_DEPLOYMENT。大规模 route 另有 Azure Speech batch、OpenAI mini transcription 与 baked local Whisper Turbo 三种强类型选择;禁止让 audio transcription 与 OCR 共享 deployment/API version。 tests/live/test_synthetic_video_ocr_service_live.py 覆盖仓库自有视频下载、ffmpeg keyframe、真实 OCR、 Job/Result 持久化、REST 读取、completion event 与 workspace 清理;它不替代 OSS 或 ASR gate。 tests/live/test_owned_sample_video_batch_live.py 以显式付费 gate 将 umbrella docs/medias/sample-vid*.mp4 五个不同视频逐一走完 HTTP fetch、ffprobe、duration-aware 单帧选择、真实 OCR、result/event 与 workspace 清理;每个视频硬限制一个唯一帧/一次 provider call,避免把验收测试变成无界模型支出。 tests/live/test_owned_sample_video_local_asr_live.py 在禁止网络的 baked Whisper image 中将同一批五个视频 走完 fetch、ffprobe、audio extraction、本地 ASR、result/event 与 workspace 清理;121.945s 音频在该 Docker CPU host 上用 161.838s 完成,识别出四个 English 与一个 Vietnamese 样本,ASR provider call 为零。 这些结果证明执行链和语言输出,不代表文字准确率已经通过 owner 的人工验收。 两个 owned-media live tests 使用 pytest -s 时都会输出无凭证的 human_review_samples,使 owner 能逐视频 检查 transcript/OCR text;测试仍只在 tmp_path 写中间文件并清理 workspace。 当前非 root production image 已使用现有认证 Mongo/Redis、typed OSS/Azure 配置启动并达到 Docker healthy;验证使用唯一 Mongo/Redis namespace,停止后已清理,且未提交媒体或写入 OSS。 RAM policy 若限制写入既有用户前缀,live gate 通过 KVE_ALIYUN_OSS_PREFIX 注入该前缀并在其下追加 唯一 phase-07-live-e2e/{run_id};测试先对不存在的唯一 key 验证 DeleteObject 权限,无法清理时在 上传前失败,且不得绕过前缀向 bucket root 写入。 共享 backend env 使用 oss-cn-shenzhen region spelling,而 Alibaba v2 signer 使用 cn-shenzhenAliyunOssSettings 在 Pydantic boundary 统一这两种输入,避免 live/runtime 产生无效签名。

规模化成本边界

当前 AsrEngine/OcrEngine Port 与 normalized Result contract 继续作为唯一分析边界。关键帧已改为按 完整 duration 抽取有界 candidates、按时间窗口 选择视觉变化最大的画面,并用 64-bit perceptual hash 去重;静态/重复画面不会再重复触发 OCR,十帧预算也 不再只覆盖视频开头十秒。OcrEngine 接收完整 ExtractedFrame,因此 OCR 与 confidence fallback 会保留 scene selection 的真实 source timestamp,不再用数组下标伪造秒数。 Dockerfile.paddle 现提供 x86_64 PP-OCRv5 Mobile mass-volume profile:一个 local batch 处理全部唯一帧, ConfidenceFallbackOcrEngine 只把空结果、未知 confidence 或低于阈值的帧发送到付费 vision adapter; base Dockerfile 保留为 compatibility/canary profile,不是海量 OCR 部署入口。PP-OCR model 在 image build 阶段缓存,production startup 会验证 inference.json/weights/config,禁止 runtime 静默下载。 模型选择与 batch predict() 边界分别依据 PP-OCRv5 official comparisonofficial OCR pipeline API。 ASR analysis artifact 使用 lossless 16 kHz mono FLAC。ffmpeg silencedetect 先建立 source-to-analysis timeline,再删除静音并保留 speech padding;ASR segment 会映射回原视频时间。若上游提供由 keble-data-infra 授权、绑定 source reference 与实际 media SHA-256 的 canonical transcript,则服务直接复用, 跳过音频提取和付费 ASR;caption 文本本身不会被误当作授权 transcript。

ASR route registry 只允许显式配置的 openai_miniazure_batchlocal_whisper,不做静默 provider fallback。Hosted default 为 JSON-only gpt-4o-mini-transcribe;没有 provider segment timestamps 时,整段 文本使用 compact audio 的完整 duration,而不会伪造零长度。Azure Speech batch 固定当前实现支持的 GA 2025-10-15;production composition 通过 canonical audio artifact_id 从 OSS 生成一小时 HTTPS signed URL, URL 不写入 Job/Result/log,且其 TTL 必须长于 bounded batch timeout。local route 固定 baked large-v3-turbo CTranslate2 artifacts,并禁止 runtime download。 三条 adapter 共用 normalized transcript、VAD timeline 和 cost estimator,但 owner 已选择 public-API-only production policy,因此最终 selector 只比较 hosted Whisper 与 Azure Speech batch;local Whisper 仅保留为 offline engineering canary。AnalysisCostBudget 在 provider call 前 fail closed:未知价格或 worst-case estimate 超预算会返回 typed ANALYSIS_BUDGET_EXCEEDED,不会先花费 再报警。

仓库自有中英文语音 fixture、manifest SHA-256、WER/CER、realtime factor 与 cost/min runner 已落地;fixture 标记为 machine_generated_pending_owner_review。Baked local Whisper route 已独立测量;hosted mini 可通过 RUN_LIVE_OPENAI_ASR=1 独立测量而不伪造 acceptance threshold。ASR-specific Azure Whisper 已在相同 EN/ZH fixtures 上通过:WER 0.0909、CER 0.06256.596s audio / 6.764s wall、realtime factor 0.975;repeat warm run 为 4.445s wall / 1.484 realtime factor。Microsoft public retail defaults are 6000 micros/audio-minute for global Azure Whisper and 3000 for standard Azure Speech batch, reviewed 2026-07-12; regional Whisper deployments override the default through env. Recommended canary thresholds are WER 0.15, CER 0.10, realtime factor 0.5, and cost ceiling 8000, but route selection still fails closed until KVE_ASR_BENCHMARK_OWNER_APPROVED=1. The authenticated two-public-route measurement now passes with automatic isolated OSS upload/sign/delete: Azure Speech Batch measured WER 0.0909, CER 0.046875, 133.34s wall, realtime factor 0.0495, and 3002 micros/audio-minute; hosted Whisper measured WER 0.0909, CER 0.0625, 5.47s wall, realtime factor 1.206, and 5995 micros/audio-minute. Both processed 6.596s of identical audio. Under the recommended short-video threshold, hosted Whisper is the only passing speed candidate and therefore remains the default. Azure Batch remains a cheaper backlog candidate only after a separate multi-file coordinator proves forecast bulk capacity; one-job-per-video queue latency must not be mislabeled as throughput. 仓库自有 OCR fixture 已在非 root amd64 Paddle image 中通过真实 PP-OCRv5 Mobile inference:KEBLE OCR 42、原始 9750 ms timestamp、provider_calls=0。Direct OpenAI and local-compute prices remain operator-owned and unknown rather than reusing Azure rates or being mislabeled free.

The owner has now approved those thresholds. A fresh configured-route run selects openai_mini with WER 0.0909, CER 0.0625, realtime factor 0.7945, and 5995 micros/audio-minute. The repository installs httpx[socks] because the OpenAI/Azure client intentionally honors the environment-owned SOCKS proxy; it must not bypass that network boundary. Azure Speech Batch remains a separately credentialed, currently unprovisioned backlog candidate—not an implicit reuse of AZURE_ENDPOINT_ASR/AZURE_KEY_ASR.

The provider-quota gate is separate from route-quality approval. Compile an operator-reviewed profile before production startup:

uv run python scripts/compile_provider_quota_profiles.py \
  docs/examples/provider-quota-profiles.example.uncompiled.json \
  /run/keble/provider-quota-profiles.json

The checked-in example intentionally records the observed 3-RPM ASR quota and therefore compiles as evidence but fails the 5,000-video capacity preflight. Do not raise that value without a provider/source record, and do not treat a missing value as unlimited.

Routing 与 gate 配置见 .env.example。不具备 ffmpeg/ffprobe 的主机使用 portable-offline 命令;依赖这些 production binaries 的 media integration modules 明确标记为 container,并在 Linux image/CI 的完整 not live suite 中执行:

uv run pytest -q -m "not live and not slow and not eval and not local_stack and not db_stack and not container"
uv run pytest -q -m eval
uv run ruff check .
npx --yes pyright .

真实 route 分别使用 RUN_LIVE_OPENAI_ASRRUN_LIVE_AZURE_SPEECH_BATCHRUN_LOCAL_WHISPER_ASR;public-API measurement 还要求 RUN_LIVE_ASR_BENCHMARK=1RUN_ALIYUN_OSS_LIVE_E2E=1。测试会上传两份 exact owned fixtures、生成 bounded signed URLs,并在 module teardown 后断言 prefix 为空;不再手工粘贴 URL。只有 production selection 额外要求显式 KVE_ASR_BENCHMARK_OWNER_APPROVED=1RUN_LIVE_OPENAI_ASR=1 uv run --env-file .env pytest -q tests/live/test_hosted_mini_asr_live.py measures only the hosted route and never selects it. The owner-approved configured-route gate is RUN_LIVE_OPENAI_ASR=1 uv run --env-file .env pytest -q -s tests/evals/test_owner_approved_hosted_asr_live.py. Gate 不会因 credential 存在而自动开启。

Base compatibility/canary image:

docker build -f Dockerfile \
  -t keble-video-enrichment:0.4.0-base .

The package-scoped release workflow publishes three immutable GHCR images from one annotated vX.Y.Z source: base hosted-provider compatibility, x86_64 Paddle OCR, and x86_64 local Whisper. Each image receives only its semantic version and source-<commit> tags, Buildx provenance, an SBOM, and a retained digest artifact; deployment consumes image@sha256:..., never latest.

gh workflow run publish-images.yaml \
  --repo keble-ai/keble-video-enrichment \
  --ref main \
  -f source_tag=v0.4.0

The accelerated distributed capacity gates are:

KVE_TEST_REDIS_URI=redis://127.0.0.1:19736/15 \
  uv run pytest -q -s tests/integration/events/test_provider_limits_real_redis.py

KVE_TEST_REDIS_URI=redis://127.0.0.1:19736/15 \
  uv run pytest -q -s tests/integration/capacity/test_video_day_replay.py

The 5,000 replay uses cached providers and accounts 15,000 audio-minutes; it is control-plane and cleanup evidence, not a claim that the 3-RPM hosted route can finish the wall-clock workload.

x86_64 local-primary image:

docker build --platform linux/amd64 -f Dockerfile.paddle \
  -t keble-video-enrichment:0.4.0-paddle .

Baked local-ASR candidate image:

docker build -f Dockerfile.whisper \
  -t keble-video-enrichment:0.4.0-whisper .

Release 0.2.0 historically resolved keble-helpers>=1.51.0,<2.0.0; the Review2 release line resolves keble-helpers>=1.57.0,<2.0.0, keble-db>=1.9.0,<2.0.0, and the mandatory keble-data-infra-contract==0.9.0 from the package registry. Docker builds use the frozen lockfile directly and require no GitHub token or Git installation. Local release verification produced base sha256:c5c248cebe56c6353ee9348d97eb7a60b31c9638c22deeda07c6ad186a5d4482; it runs as non-root UID 10001, contains service 0.4.0, contract 0.9.0, helper 1.54.0, and database 1.9.0, and has no Git client or credential plumbing. The same image passed all 14 ffmpeg-backed media integration cases.

Dockerfile.whisper pins the MIT CTranslate2 conversion dropbox-dash/faster-whisper-large-v3-turbo at immutable revision 0a363e9161cbc7ed1431c9597a8ceaf0c4f78fcf. Build time downloads the six approved files, writes their SHA-256 manifest, verifies it, and makes /opt/faster-whisper read-only. Startup rehashes the artifact set before loading with local_files_only=True; runtime never downloads weights. This is the independently measurable local-ASR candidate, not an automatic production winner and not a replacement for the PP-OCR mass-volume image.

Apple Silicon host 可运行 unit 与真实 macOS local canary,但 Paddle 3.3.0 没有 Linux arm64 PyPI wheel; 因此 production image 必须在 x86_64 builder 构建,或由 CI 使用 --platform linux/amd64 构建。 历史 base/Paddle ASR-composition images 为 sha256:36ce878d... / sha256:262bf83a...。 历史 Mongo runtime candidate 是从 exact 9db27c8 lock 重建的 Paddle image sha256:eae43a9a8faec480d35986d9c9e77f08036233e8ee5ed42443ed3dbb6be1d8ca;它以 non-root kve 运行、不含 Git 或 /root/.gitconfig,并已通过真实 Job POST -> GET。 Paddle image 的 Docker uncompressed size 为约 3.18 GB(Paddle runtime + OpenCV + ffmpeg);这是 部署/cold-start 成本,不得把“provider call 为 0”误解成“总成本为 0”。正式放量前仍需比较常驻 worker amortization、smaller inference runtime 和 hosted OCR 的总拥有成本。

旧 image digest 是 ASR route 组合 smoke 的历史证据:完整服务链曾证明 registry keble-db 1.8.1 会把 BSON datetime 解码为 naive 值并令严格 Job 读取失败。Release 0.1.1 已切换到包含同一 UTC 修复的 registry keble-db 1.9.0;所有后续 production image 必须从当前 lock 重建并通过真实 Job POST -> GET,禁止仅以 /health 代替持久化读取证明。

状态权威表: Master Plan §24

合规 gate

授权(OD-00)通过前:禁止下载/处理/保存任何真实第三方视频;允许契约、文档、本地合成视频、 脱敏 fixture、单元测试与本地 HTTP server 集成测试。 详见 架构文档 §10open-decisions

文档入口

Composer 执行入口

  1. composer-execution-guide.md
  2. Phase 07 应用开发已完成;外部验证入口见 Phase 07Operations Runbook

当前是否允许进入编码阶段

Phase 05/06 已完成;Phase 07 application development 已完成;OD-01/OD-02/OD-06 均为 DECIDED Data Platform 双入口保持 owner-approved、DEFERRED、NON-BLOCKING。剩余工作是 gated Aliyun OSS、生产 Mongo/Redis/provider 和目标服务器部署验证,不是未完成的应用编码。

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

keble_video_enrichment-0.4.0.tar.gz (708.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

keble_video_enrichment-0.4.0-py3-none-any.whl (236.5 kB view details)

Uploaded Python 3

File details

Details for the file keble_video_enrichment-0.4.0.tar.gz.

File metadata

  • Download URL: keble_video_enrichment-0.4.0.tar.gz
  • Upload date:
  • Size: 708.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.10.9 {"installer":{"name":"uv","version":"0.10.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for keble_video_enrichment-0.4.0.tar.gz
Algorithm Hash digest
SHA256 a43e33933c6431b2f52d0de6233af21b8b9bc1b8178da49da0f0165dc56e09ff
MD5 e8c3ac24a07b2b9fb2e0abd30ed2f51a
BLAKE2b-256 f003205ddc9dc717de32462df4c1a18bb9ea7117026ca903cf67354195528638

See more details on using hashes here.

File details

Details for the file keble_video_enrichment-0.4.0-py3-none-any.whl.

File metadata

  • Download URL: keble_video_enrichment-0.4.0-py3-none-any.whl
  • Upload date:
  • Size: 236.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.10.9 {"installer":{"name":"uv","version":"0.10.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for keble_video_enrichment-0.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 09929d8dca2b4b00705402616136a424c42c6d616fc5bbcd99d6a8396ffda26f
MD5 de366333f23dba57a4e76a513b74ba8b
BLAKE2b-256 c2d2148f06540bacc99584cf8d5bea335fbb53141488ac11152759c34a856b82

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page