isagellm-protocol 0.4.0.6

sageLLM protocol types and validation (Protocol v0.1)

sagellm-protocol

Protocol Compliance (Mandatory)

• This repository defines Protocol v0.1 as the source of truth.
• Any globally shared definitions (fields, error codes, metrics, IDs, schemas) MUST be added here before use.

Type definitions and validation for sageLLM inference engine.

• Package: isagellm-protocol
• Import: sagellm_protocol
• Python: 3.10+

Installation

pip install isagellm-protocol

For development:

git clone [e-mail]:intellistream/sagellm-protocol.git
cd sagellm-protocol
pip install -e ".[dev]"

Features

• Request/Response type definitions
• Streaming event types (start/delta/end)
• Performance metrics with validation
• Error types and error codes
• Timestamp tracking
• KV cache lifecycle hooks
• Pydantic v2 validation

Quick Start

from sagellm_protocol import Request, Response, Metrics, ErrorCode

# 创建请求
req = Request(
    request_id="req-001",
    trace_id="trace-001",
    model="llama2-7b",
    prompt="Hello, world!",
    max_tokens=128,
    stream=False,
    temperature=0.7,
)

# 创建响应
metrics = Metrics(
    ttft_ms=45.2,
    tbt_ms=12.5,
    throughput_tps=80.0,
    peak_mem_mb=24576,
    error_rate=0.0,
)

resp = Response(
    request_id="req-001",
    trace_id="trace-001",
    output_text="Hi there!",
    output_tokens=[42, 17],
    finish_reason="stop",
    metrics=metrics,
)

For more examples, see examples/basic_usage.py.

API Reference

Core Types

• Request / Response - Request and response objects
• Metrics - Performance metrics
• StreamEvent - Streaming event types
• Error / ErrorCode - Error handling
• Timestamps - Timestamp tracking
• KVAllocateParams / KVHandle - KV cache management

Development

Run tests:

pytest tests/ -v

Format and lint:

ruff format .
ruff check . --fix

Type check:

mypy src/sagellm_protocol

🔄 贡献指南

请遵循以下工作流程：

1. 创建 Issue - 描述问题/需求

   gh issue create --title "[Bug] 描述" --label "bug,sagellm-protocol"
   

2. 开发修复 - 在本地 fix/#123-xxx 分支解决

   git checkout -b fix/#123-xxx origin/main-dev
   # 开发、测试...
   pytest -v
   ruff format . && ruff check . --fix
   

3. 发起 PR - 提交到 main-dev 分支

   gh pr create --base main-dev --title "Fix: 描述" --body "Closes #123"
   

4. 合并 - 审批后合并到 main-dev

更多详情见 .github/copilot-instructions.md

Documentation

For more details:

• Testing Guide
• Contributing Guidelines

License

Proprietary - IntelliStream