Skip to main content

ei-pipe-sdk

面向算法同学的极简模型接入 SDK。把你的模型写成一个 EiInfer 的两个方法,ei-pipeline 就能调度它推理:你不用关心数据怎么进来、结果怎么提交,也不用关心部署。

安装

pip install ei-pipe-sdk
# 或先在仓库 ei_pipe_sdk/ 下本地构建
uv build && pip install dist/ei_pipe_sdk-*.whl

三步接入

# my_model.py
from ei_infer import EiInfer, register


class MyModel(EiInfer):
    def init(self, model_cfg: dict | None = None) -> None:
        # 进程启动时调用一次:加载权重、tokenizer、选设备。失败会让 worker 启动即崩溃。
        self.model = load_weights(...)

    def inference(self, input_path: str, output_path: str, node_info: dict) -> dict:
        # input_path / output_path: worker 已恢复好的绝对路径
        #   = {data_root}/{service_id}/buckets/{time_bucket}/{pipeline_id}/<相对路径>
        # input_path 是输入(文件或目录);output_path 是允许写结果文件的目录(可能为空)。
        # 是否落盘由模型自己决定;返回的 dict 会被 worker 回传给 pipeline。
        return {"detections": self.model(open(input_path))}


register("my-model", MyModel())   # 注册名 = 管道节点 params.name 的值
pip install ei-pipe-sdk
# worker 入口:参数经构造参数显式给定(也可经配置文件,见下文),
# SDK 不读任何环境变量。每个注册模型各消费一条自己的队列。
from ei_infer import InferNode

node = InferNode(
    redis_url="redis://scheduler:6379/0",
    service_id="3fa2b1c9",  # 调度服务 h8;或经配置文件给 scheduler_url 解析
    data_root="/lpai/pvc/ei-autolabel-bd-ga-infer",  # PFS 挂载点裸路径
)
node.run()

协议

调度端(ei-pipeline 的 ei_infer 节点)会按统一协议给你发任务:

字段 类型 说明
name str 模型注册名,由你 register("name", ...) 时命名
input_path str 输入路径(相对 pipe 共享目录,如 data/),worker 恢复为绝对路径
output_path str? 允许写结果文件的目录(相对 pipe 共享目录,可选)
version str? 版本提示,透传保留

worker 用 node_info.service_id + time_bucket + pipeline_id 与 InferNode 配置的 data_root 把 input_path / output_path 恢复成绝对路径 ({data_root}/{service_id}/buckets/{time_bucket}/{pipeline_id}/<相对路径>), 调用 inference(input_path, output_path, node_info[, abort_event])。 是否把结果落盘由模型自己决定,worker 不替模型落盘,只把模型返回的 dict 原样回传。 整个链路里你的函数只看绝对路径 input_path / output_path 和 node_info,其余(数据就位、 并发、轮询)全部由 SDK 兜底。

协作式取消

Python 线程杀不掉。job 被取消(job.abort())或超时时,worker 会激活该 job 专属的 abort_event;inference 声明第 4 个参数即可收到它,在推理间隙检查并 提前退出:

from ei_infer import EiInfer, InferenceAborted


class MyModel(EiInfer):
    def inference(self, input_path, output_path, node_info, abort_event=None):
        for item in load_items(input_path):
            if abort_event.is_set():  # 推理间隙检查取消信号
                raise InferenceAborted("job cancelled")
            ...
  • 事件按 job 生成,并发执行时各 job 各持独立事件,互不影响。
  • 老的 3 参签名不受影响(拿不到取消信号,跑到自然结束),完全向后兼容。
  • 事件未激活时自发抛 InferenceAborted 按普通 job 失败处理。

配置

参数一律显式给定(InferNode 构造参数,或经配置文件 + build_node), SDK 不读任何环境变量。

构造参数 / 配置文件字段:

字段 默认 说明
redis_url 必填 消费队列的 Redis(与调度服务同一套)
service_id 二选一 调度服务的 service_id(h8),静态给定时优先生效
scheduler_url 二选一 未给 service_id 时,向该调度请求 GET /api/status/service-id 解析(不可达每 10 秒重试)
data_root 必填 PFS 挂载点裸路径;构建时自动并入 {service_id}/buckets 隔离段
max_jobs 4 每模型队列的并发任务数(arq max_jobs)
max_threads max(4, max_jobs) 跑同步推理的线程池大小(仅构造参数)
result_max_bytes 65536 job result 进 Redis 前的尺寸上限(字节);超限 job 失败(ei_arq.result_guard)

配置文件(yaml/json,{{变量名}} 占位由 secrets 映射在载入时注入)::

service_id: '{{service_id}}'
# scheduler_url: http://scheduler:8000
redis_url: redis://localhost:6379/0
data_root: /lpai/pvc/ei-autolabel-bd-ga-infer
max_jobs: 4
from ei_infer.worker import build_node

node = build_node("infer.yaml", secrets={"service_id": "3fa2b1c9"})
node.run()

完整示例见 example/(infer_exa.py + run.sh)。示例只需三个变量 (redis、data_root、service_id/scheduler_url),直接构造 InferNode, 不走配置文件;配置文件方式用于生产部署。

队列

每个注册模型一条队列,统一文法(详见 ei-pipeline docs/arq_queue_naming.md):

arq:ei:{service_id}:ei_infer:{model_name}

worker 进程为每个注册模型起一个 arq Worker 消费对应队列;service_id 与 调度侧表名/Redis 前缀同源,隔离共用一套 Redis 的多个调度服务。

多模型

多个模型可在同一个 worker 进程里 register 多个名字;进程会为每个模型各起 一个 arq Worker,各自消费自己的队列,互不抢任务。

失败模式

  • inference 返回非 dict → 任务失败(错误信息含模型名)。
  • name 未注册 → 任务失败,节点错误信息会指出未知模型名。
  • init 抛异常 → worker 启动即退出,方便在你自己的日志里早发现。
  • inference 抛 InferenceAborted:abort 事件已激活 → 任务取消;未激活 → 普通任务失败。

本地联调

# 起一个本地 redis 后(redis-server),在 example/ 下:
bash run.sh   # infer_exa.py(前台)+ ois_exa.py(后台)
# 每个注册模型一条队列:arq:ei:{service_id}:ei_infer:{model_name}

然后在 ei-pipeline 用 ei_infer 节点提交一条任务即可走通。

Metadata

Release files for ei-pipe-sdk 0.2.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ei-pipe-sdk 0.2.10
File Size Uploaded Provenance Actions
ei_pipe_sdk-0.2.10.tar.gz 99.4 kB Without provenance Details for ei_pipe_sdk-0.2.10.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ei-pipe-sdk 0.2.10
File Interpreter ABI Platform Actions
ei_pipe_sdk-0.2.10-py3-none-any.whl Python 3 none any Details for ei_pipe_sdk-0.2.10-py3-none-any.whl

Total release size: 182.9 kB

Release files / ei_pipe_sdk-0.2.10.tar.gz

Download URL ei_pipe_sdk-0.2.10.tar.gz
Size 99.4 kB
Tags Source
SHA-256 checksum
How to use checksums
24f2ff2acd8790d790393b466ef8868fcf3710cbd901b1882669fae9803cb316
BLAKE2b-256 checksum
How to use checksums
882c21dd65929b5d244249b482516ef2a10a892dcb9ddca69aaf7f18f7defc2e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / ei_pipe_sdk-0.2.10-py3-none-any.whl

Download URL ei_pipe_sdk-0.2.10-py3-none-any.whl
Size 83.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a9bfc1aed6b620c5d8519cbcc31cd4401c8fd6a811cdb89045d1b8f4c01e928a
BLAKE2b-256 checksum
How to use checksums
5fc1051ebf8b490066ecdf0e2e95b92eada72d935d343113cc121b57e8661c38
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Security for ei-pipe-sdk 0.2.10

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page