sageLLM Gateway
Protocol Compliance (Mandatory)
- MUST follow Protocol v0.1: https://github.com/intellistream/sagellm-docs/blob/main/docs/specs/protocol_v0.1.md
- Any globally shared definitions (fields, error codes, metrics, IDs, schemas) MUST be added to Protocol first.
isagellm-gateway - OpenAI/Anthropic Compatible API Gateway for sageLLM
Overview
The isagellm-gateway package serves as the primary entry point for external API requests in the
sageLLM architecture. It provides an OpenAI-compatible interface and handles session management,
request routing, and integration with the sageLLM Control Plane.
Key Features:
- OpenAI Compatibility: Full support for
/v1/chat/completionsand/v1/models. - Session Management: Built-in support for conversational sessions via
/sessionsendpoints. - Control Plane Integration: Seamlessly connects with
isagellm-control-planefor model management and routing. - Distributed Tracing: Built-in OpenTelemetry support for observability across services (Jaeger/Zipkin compatible).
- Protocol Compliance: Strictly adheres to sageLLM Protocol v0.1.
Dependencies
- Python: >= 3.10
- isagellm-protocol: >= 0.4.0.0
- isagellm-control-plane: Required for runtime operation.
Installation
Install the gateway and the required control plane package:
pip install isagellm-gateway isagellm-control-plane
Quick Start
Start the Server
Start the gateway server on port 8080:
sagellm-gateway --port 8080
Usage Examples
1. Chat Completion (OpenAI API)
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gpt2",
"messages": [{"role": "user", "content": "Hello!"}]
}'
2. List Models
curl http://localhost:8080/v1/models
3. Health Check
curl http://localhost:8080/health
API Documentation
Core Endpoints
POST /v1/chat/completions: Generate chat completions (supports streaming).POST /v1/embeddings: Generate embeddings.GET /v1/models: List available models.GET /health: Health check and status.
Session Management
POST /sessions: Create a new session.GET /sessions: List all sessions.GET /sessions/{session_id}: Get session details.DELETE /sessions/{session_id}: Delete a session.
Architecture
┌─────────────────────────────────────────┐
│ sageLLM Gateway │
│ • OpenAI-compatible API │
│ • Session State Management │
│ • Request Validation & Routing │
└─────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ Control Plane │
│ (managed by isagellm-control-plane) │
└─────────────────────────────────────────┘
Development Guide
Prerequisites
- Python 3.10+
- Git
Setup
-
Clone the repository:
git clone git@github.com:intellistream/sagellm-gateway.git cd sagellm-gateway
-
Initialize development environment:
# 开发模式(默认):先装 PyPI,再用本地 editable 覆盖
./quickstart.sh --dev
# 标准模式:仅使用 PyPI 稳定包
./quickstart.sh --standard
# 查看帮助(模式说明与参数)
./quickstart.sh --help
# Note: You also need the control plane for local testing
pip install isagellm-control-plane
安装脚本行为说明:
- 安装前会动态清理同前缀历史安装包(默认
isagellm*) --dev模式下本地 editable 覆盖使用--no-deps,避免重复依赖解析- 不会创建 venv,复用当前已配置的非-venv Python 环境
Testing
Run the test suite with pytest:
pytest tests/ -v
Linting
Check code quality with ruff:
ruff check .
Actions 账单阻塞时的本地替代 CI
当 GitHub Actions 因 billing/quota 无法启动时,在仓库根目录运行:
bash scripts/local_ci_fallback.sh
该脚本覆盖 ci.yml 与 version-source-guard.yml 的核心检查路径,便于本地复现与验收。
Versioning
Current Version: 0.4.0.11
For version history, see CHANGELOG.md.
Metadata
Release files for isagellm-gateway 0.5.4.14
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| isagellm_gateway-0.5.4.14.tar.gz | 110.2 kB | Without provenance | Details for isagellm_gateway-0.5.4.14.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| isagellm_gateway-0.5.4.14-py2.py3-none-any.whl | Python 2, Python 3 | none | any | Details for isagellm_gateway-0.5.4.14-py2.py3-none-any.whl |
Total release size: 237.1 kB
Release files / isagellm_gateway-0.5.4.14.tar.gz
| Download URL | isagellm_gateway-0.5.4.14.tar.gz |
|---|---|
| Size | 110.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
922f1a8d14cb460f41db918bc7df64873784eab134c00a2455475f6c43add224
|
|
BLAKE2b-256 checksum How to use checksums |
fb6e8795a585a8f0c5f263ccf94d008d1aba02ce1f3ff94dc20e831db2a85760
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.15
|
Release files / isagellm_gateway-0.5.4.14-py2.py3-none-any.whl
| Download URL | isagellm_gateway-0.5.4.14-py2.py3-none-any.whl |
|---|---|
| Size | 126.9 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
171805107fdaf2c49b847fa54ba7dffb219fa43bedfff8e99fd61b9e6b9009bc
|
|
BLAKE2b-256 checksum How to use checksums |
0c9f3d647d2bad0229d5a5f9014b5369b892db284c8453b18ff1d999779c5d51
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.15
|
Security for isagellm-gateway 0.5.4.14
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.