sageLLM Gateway

Protocol Compliance (Mandatory)

MUST follow Protocol v0.1: https://github.com/intellistream/sagellm-docs/blob/main/docs/specs/protocol_v0.1.md
Any globally shared definitions (fields, error codes, metrics, IDs, schemas) MUST be added to Protocol first.

isagellm-gateway - OpenAI/Anthropic Compatible API Gateway for sageLLM

Overview

The isagellm-gateway package serves as the primary entry point for external API requests in the sageLLM architecture. It provides an OpenAI-compatible interface and handles session management, request routing, and integration with the sageLLM Control Plane.

Key Features:

OpenAI Compatibility: Full support for /v1/chat/completions and /v1/models.
Session Management: Built-in support for conversational sessions via /sessions endpoints.
Control Plane Integration: Seamlessly connects with isagellm-control-plane for model management and routing.
Distributed Tracing: Built-in OpenTelemetry support for observability across services (Jaeger/Zipkin compatible).
Protocol Compliance: Strictly adheres to sageLLM Protocol v0.1.

Dependencies

Python: >= 3.10
isagellm-protocol: >= 0.4.0.0
isagellm-control-plane: Required for runtime operation.

Installation

Install the gateway and the required control plane package:

pip install isagellm-gateway isagellm-control-plane

Quick Start

Start the Server

Start the gateway server on port 8080:

sagellm-gateway --port 8080

Usage Examples

1. Chat Completion (OpenAI API)

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt2",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

2. List Models

curl http://localhost:8080/v1/models

3. Health Check

curl http://localhost:8080/health

API Documentation

Core Endpoints

POST /v1/chat/completions: Generate chat completions (supports streaming).
POST /v1/embeddings: Generate embeddings.
GET /v1/models: List available models.
GET /health: Health check and status.

Session Management

POST /sessions: Create a new session.
GET /sessions: List all sessions.
GET /sessions/{session_id}: Get session details.
DELETE /sessions/{session_id}: Delete a session.

Architecture

┌─────────────────────────────────────────┐
│         sageLLM Gateway                 │
│  • OpenAI-compatible API                │
│  • Session State Management             │
│  • Request Validation & Routing         │
└─────────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────┐
│         Control Plane                   │
│  (managed by isagellm-control-plane)    │
└─────────────────────────────────────────┘

Development Guide

Prerequisites

Python 3.10+
Git

Setup

Clone the repository:

git clone git@github.com:intellistream/sagellm-gateway.git
cd sagellm-gateway

Initialize development environment:

# 开发模式（默认）：先装 PyPI，再用本地 editable 覆盖
./quickstart.sh --dev

# 标准模式：仅使用 PyPI 稳定包
./quickstart.sh --standard

# 查看帮助（模式说明与参数）
./quickstart.sh --help

 # Note: You also need the control plane for local testing
 pip install isagellm-control-plane

安装脚本行为说明：

安装前会动态清理同前缀历史安装包（默认 isagellm*）
--dev 模式下本地 editable 覆盖使用 --no-deps，避免重复依赖解析
不会创建 venv，复用当前已配置的非-venv Python 环境

Testing

Run the test suite with pytest:

pytest tests/ -v

Linting

Check code quality with ruff:

ruff check .

Actions 账单阻塞时的本地替代 CI

当 GitHub Actions 因 billing/quota 无法启动时，在仓库根目录运行：

bash scripts/local_ci_fallback.sh

该脚本覆盖 ci.yml 与 version-source-guard.yml 的核心检查路径，便于本地复现与验收。

Versioning

Current Version: 0.4.0.11

For version history, see CHANGELOG.md.

Metadata

Release files for isagellm-gateway 0.5.4.14

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for isagellm-gateway 0.5.4.14
File	Size	Uploaded
isagellm_gateway-0.5.4.14.tar.gz	110.2 kB	Mar 12, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for isagellm-gateway 0.5.4.14
File	Interpreter	ABI	Platform	Reset
isagellm_gateway-0.5.4.14-py2.py3-none-any.whl 126.9 kB Mar 12, 2026	Python 3, Python 2	none	any	Details

Total release size: 237.1 kB

Release files / isagellm_gateway-0.5.4.14.tar.gz

Download URL	isagellm_gateway-0.5.4.14.tar.gz
Size	110.2 kB
Tags	Source
SHA-256 checksum How to use checksums	`922f1a8d14cb460f41db918bc7df64873784eab134c00a2455475f6c43add224`
BLAKE2b-256 checksum How to use checksums	`fb6e8795a585a8f0c5f263ccf94d008d1aba02ce1f3ff94dc20e831db2a85760`
Upload date	Mar 12, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.11.15`

Release files / isagellm_gateway-0.5.4.14-py2.py3-none-any.whl

Download URL	isagellm_gateway-0.5.4.14-py2.py3-none-any.whl
Size	126.9 kB
Tags	Python 2 Python 3
SHA-256 checksum How to use checksums	`171805107fdaf2c49b847fa54ba7dffb219fa43bedfff8e99fd61b9e6b9009bc`
BLAKE2b-256 checksum How to use checksums	`0c9f3d647d2bad0229d5a5f9014b5369b892db284c8453b18ff1d999779c5d51`
Upload date	Mar 12, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.11.15`