isagellm-control-plane 0.5.0.0

sageLLM Control Plane - Intelligent request routing, scheduling, and engine lifecycle management

sageLLM Control Plane

Protocol Compliance (Mandatory)

• MUST follow Protocol v0.1: https://github.com/intellistream/sagellm-docs/blob/main/docs/specs/protocol_v0.1.md
• Any globally shared definitions (fields, error codes, metrics, IDs, schemas) MUST be added to Protocol first.

Intelligent request routing, scheduling, and engine lifecycle management for sageLLM.

职责定位

Control Plane 位于用户/Gateway 与执行引擎之间，负责：

• 注册与健康管理引擎
• 请求路由与调度（含 PD 分离场景）
• 负载均衡与基础扩缩容钩子

依赖关系

PyPI 包名：isagellm-control-plane ｜ 导入命名空间：sagellm_control

依赖（以 pyproject.toml 为准）：

• isagellm-protocol>=0.4.0.0,<0.5.0
• pydantic>=2.0.0
• httpx>=0.24.0

可选（用于本地直接执行引擎）：

• isagellm-core（提供 LLMEngine）

被以下组件使用（示例）：

• sagellm（统一入口/CLI）
• sagellm-gateway（API 网关）

安装指南

pip install isagellm-control-plane

Requirements: Python 3.10+

快速开始（CPU-first，可运行）

import asyncio

from sagellm_control import ControlPlaneManager, EngineState, ExecutionInstanceType
from sagellm_protocol import Request


async def main() -> None:
    cp = ControlPlaneManager(scheduling_policy="fifo", routing_strategy="least_loaded", mode="local")

    cp.register_engine(
        engine_id="engine-001",
        model_id="Qwen2-7B",
        host="localhost",
        port=8001,
        engine_kind="llm",
        metadata={"instance_type": ExecutionInstanceType.GENERAL.value},
    )
    cp.update_engine_state("engine-001", EngineState.READY)

    req = Request(
        request_id="req-001",
        trace_id="trace-001",
        model="Qwen2-7B",
        prompt="Hello",
        max_tokens=16,
        stream=False,
    )

    decision = await cp.schedule_request(
        request_id=req.request_id,
        trace_id=req.trace_id,
        model_id=req.model,
        prompt=req.prompt,
        max_tokens=req.max_tokens,
    )
    print(decision)

    cp.unregister_engine("engine-001")


if __name__ == "__main__":
    asyncio.run(main())

完整演示见 examples/mvp_integration_demo.py。

Scheduler IR 模块使用说明

Control Plane 内置 Scheduler IR（Intermediate Representation）模块，用于把请求调度过程表达为可优化图结构：

• IRBuilder：把请求构建为 SchedulerIR（Task/Prefill/Decode 节点 + 依赖边）
• IROptimizer：执行可插拔优化 Pass（如 KVReusePass、ComputeCommOverlapPass）
• DefaultIRExecutor：把 IR 翻译为执行命令并通过 Control Plane/Engine Client 执行

可直接从根包导入：

from sagellm_control import IRBuilder, IROptimizer, DefaultIRExecutor

示例程序（至少 3 个）：

• 基础构建：examples/ir_basic_example.py
• 优化流程：examples/ir_optimization_example.py
• KV-aware 场景：examples/ir_kv_aware_example.py

运行示例：

python examples/ir_basic_example.py
python examples/ir_optimization_example.py
python examples/ir_kv_aware_example.py

API 文档（核心接口）

• ControlPlaneManager
  • register_engine() / unregister_engine() / list_engines()
  • schedule_request()
  • execute_request() / stream_request()
  • get_embeddings()
• EngineClient（HTTP 调用执行引擎）
• LocalEngineClient（本地直接调用引擎）
• Scheduler IR：SchedulerIR, IRBuilder, IROptimizer, DefaultIRExecutor
• SchedulingPolicy 及内置策略（FIFOPolicy, PriorityPolicy, SLOAwarePolicy, AdaptivePolicy, KVAwareSchedulingPolicy）
• 关键类型：EngineInfo, EngineState, SchedulingDecision, RequestPriority, RequestType

架构图示

flowchart LR
    A[Gateway/Client] --> B[ControlPlaneManager]
    B --> C[SchedulingPolicy]
    B --> D[RequestRouter/LoadBalancer]
    B --> E[EngineLifecycleManager]
    D --> F[EngineClient (HTTP)]
    B --> G[LocalEngineClient]
    F --> H[Execution Engines]
    G --> H

代码结构

sagellm_control/
├── types.py             # Core data types (EngineInfo, SchedulingDecision, etc.)
├── policies/            # Scheduling policies (FIFO, Priority, SLO-aware, Adaptive)
├── router.py            # Request routing and load balancing
├── lifecycle.py         # Engine lifecycle management
├── scaling.py           # Scaling manager (MVP hooks)
├── engine_client.py     # HTTP client to engines
├── local_engine_client.py # Local (in-process) engine client
├── ir/                  # Scheduler IR (types/builder/optimizer/executor)
└── manager.py           # ControlPlaneManager

开发指南

git clone [e-mail]:intellistream/sagellm-control-plane.git
cd sagellm-control-plane
./quickstart.sh

# 或手动安装
pip install -e ".[dev]"

运行测试：

pytest tests/ -v

Lint/格式化：

ruff format .
ruff check . --fix

提交流程：

• 创建 Issue
• 在 fix/#123-xxx 分支开发
• 提交 PR 到 main-dev

版本信息

• 当前版本：0.5.0.0
• 变更记录：CHANGELOG.md

Related Repositories

• sagellm - Umbrella 包 + CLI
• sagellm-protocol - 协议定义
• sagellm-backend - 后端抽象
• sagellm-gateway - API 网关

---

License

Proprietary - IntelliStream