isagellm-protocol 0.3.0.1

sageLLM protocol types and validation (Protocol v0.1)

sagellm-protocol

Protocol Compliance (Mandatory)

• This repository defines Protocol v0.1 as the source of truth.
• Any globally shared definitions (fields, error codes, metrics, IDs, schemas) MUST be added here before use.

Type definitions and validation for sageLLM inference engine.

• Package: isagellm-protocol
• Import: sagellm_protocol
• Python: 3.10+

Installation

pip install isagellm-protocol

For development:

git clone [e-mail]:intellistream/sagellm-protocol.git
cd sagellm-protocol
pip install -e ".[dev]"

Features

• Request/Response type definitions
• Streaming event types (start/delta/end)
• Performance metrics with validation
• Error types and error codes
• Timestamp tracking
• KV cache lifecycle hooks
• Pydantic v2 validation

Quick Start

from sagellm_protocol import Request, Response, Metrics, ErrorCode

# 创建请求
req = Request(
    request_id="req-001",
    trace_id="trace-001",
    model="llama2-7b",
    prompt="Hello, world!",
    max_tokens=128,
    stream=False,
    temperature=0.7,
)

# 创建响应
metrics = Metrics(
    ttft_ms=45.2,
    tbt_ms=12.5,
    throughput_tps=80.0,
    peak_mem_mb=24576,
    error_rate=0.0,
)

resp = Response(
    request_id="req-001",
    trace_id="trace-001",
    output_text="Hi there!",
    output_tokens=[42, 17],
    finish_reason="stop",
    metrics=metrics,
)

For more examples, see examples/basic_usage.py.

API Reference

Core Types

• Request / Response - Request and response objects
• Metrics - Performance metrics
• StreamEvent - Streaming event types
• Error / ErrorCode - Error handling
• Timestamps - Timestamp tracking
• KVAllocateParams / KVHandle - KV cache management

Development

Run tests:

pytest tests/ -v

Format and lint:

ruff format .
ruff check . --fix

Type check:

mypy src/sagellm_protocol

Documentation

For more details:

• Testing Guide
• Contributing Guidelines

License

Proprietary - IntelliStream