sageLLM Gateway
Protocol Compliance (Mandatory)
- MUST follow Protocol v0.1: https://github.com/intellistream/sagellm-docs/blob/main/docs/specs/protocol_v0.1.md
- Any globally shared definitions (fields, error codes, metrics, IDs, schemas) MUST be added to Protocol first.
isagellm-gateway - OpenAI/Anthropic Compatible API Gateway for sageLLM
Overview
The isagellm-gateway package serves as the primary entry point for external API requests in the
sageLLM architecture. It provides an OpenAI-compatible interface and handles session management,
request routing, and integration with the sageLLM Control Plane.
Key Features:
- OpenAI Compatibility: Full support for
/v1/chat/completionsand/v1/models. - Session Management: Built-in support for conversational sessions via
/sessionsendpoints. - Control Plane Integration: Seamlessly connects with
isagellm-control-planefor model management and routing. - Distributed Tracing: Built-in OpenTelemetry support for observability across services (Jaeger/Zipkin compatible).
- Protocol Compliance: Strictly adheres to sageLLM Protocol v0.1.
Dependencies
- Python: >= 3.10
- isagellm-protocol: >= 0.4.0.0
- isagellm-control-plane: Required for runtime operation.
Installation
Install the gateway and the required control plane package:
pip install isagellm-gateway isagellm-control-plane
Quick Start
Start the Server
Start the gateway server on port 8080:
sagellm-gateway --port 8080
Usage Examples
1. Chat Completion (OpenAI API)
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gpt2",
"messages": [{"role": "user", "content": "Hello!"}]
}'
2. List Models
curl http://localhost:8080/v1/models
3. Health Check
curl http://localhost:8080/health
API Documentation
Core Endpoints
POST /v1/chat/completions: Generate chat completions (supports streaming).POST /v1/embeddings: Generate embeddings.GET /v1/models: List available models.GET /health: Health check and status.
Session Management
POST /sessions: Create a new session.GET /sessions: List all sessions.GET /sessions/{session_id}: Get session details.DELETE /sessions/{session_id}: Delete a session.
Architecture
┌─────────────────────────────────────────┐
│ sageLLM Gateway │
│ • OpenAI-compatible API │
│ • Session State Management │
│ • Request Validation & Routing │
└─────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ Control Plane │
│ (managed by isagellm-control-plane) │
└─────────────────────────────────────────┘
Development Guide
Prerequisites
- Python 3.10+
- Git
Setup
-
Clone the repository:
git clone git@github.com:intellistream/sagellm-gateway.git cd sagellm-gateway
-
Initialize development environment:
# 开发模式（默认）：先装 PyPI，再用本地 editable 覆盖
./quickstart.sh --dev
# 标准模式：仅使用 PyPI 稳定包
./quickstart.sh --standard
# 查看帮助（模式说明与参数）
./quickstart.sh --help
# Note: You also need the control plane for local testing
pip install isagellm-control-plane
安装脚本行为说明：
- 安装前会动态清理同前缀历史安装包（默认
isagellm*） --dev模式下本地 editable 覆盖使用--no-deps，避免重复依赖解析- 不会创建 venv，复用当前已配置的非-venv Python 环境
Testing
Run the test suite with pytest:
pytest tests/ -v
Linting
Check code quality with ruff:
ruff check .
Actions 账单阻塞时的本地替代 CI
当 GitHub Actions 因 billing/quota 无法启动时，在仓库根目录运行：
bash scripts/local_ci_fallback.sh
该脚本覆盖 ci.yml 与 version-source-guard.yml 的核心检查路径，便于本地复现与验收。
Versioning
Current Version: 0.4.0.11
For version history, see CHANGELOG.md.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file isagellm_gateway-0.5.4.14.tar.gz.
File metadata
- Download URL: isagellm_gateway-0.5.4.14.tar.gz
- Upload date:
- Size: 110.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.11.15
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
922f1a8d14cb460f41db918bc7df64873784eab134c00a2455475f6c43add224
|
|
| MD5 |
5f40cbc7afea8e439541772b3d310d33
|
|
| BLAKE2b-256 |
fb6e8795a585a8f0c5f263ccf94d008d1aba02ce1f3ff94dc20e831db2a85760
|
File details
Details for the file isagellm_gateway-0.5.4.14-py2.py3-none-any.whl.
File metadata
- Download URL: isagellm_gateway-0.5.4.14-py2.py3-none-any.whl
- Upload date:
- Size: 126.9 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.11.15
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
171805107fdaf2c49b847fa54ba7dffb219fa43bedfff8e99fd61b9e6b9009bc
|
|
| MD5 |
8208a5d5e3703ea5f77b59d75ac02fa1
|
|
| BLAKE2b-256 |
0c9f3d647d2bad0229d5a5f9014b5369b892db284c8453b18ff1d999779c5d51
|
