isagellm-gateway 0.5.4.14

sageLLM Gateway - OpenAI/Anthropic compatible API Gateway (L6)

sageLLM Gateway

Protocol Compliance (Mandatory)

• MUST follow Protocol v0.1: https://github.com/intellistream/sagellm-docs/blob/main/docs/specs/protocol_v0.1.md
• Any globally shared definitions (fields, error codes, metrics, IDs, schemas) MUST be added to Protocol first.

isagellm-gateway - OpenAI/Anthropic Compatible API Gateway for sageLLM

Overview

The isagellm-gateway package serves as the primary entry point for external API requests in the sageLLM architecture. It provides an OpenAI-compatible interface and handles session management, request routing, and integration with the sageLLM Control Plane.

Key Features:

• OpenAI Compatibility: Full support for /v1/chat/completions and /v1/models.
• Session Management: Built-in support for conversational sessions via /sessions endpoints.
• Control Plane Integration: Seamlessly connects with isagellm-control-plane for model management and routing.
• Distributed Tracing: Built-in OpenTelemetry support for observability across services (Jaeger/Zipkin compatible).
• Protocol Compliance: Strictly adheres to sageLLM Protocol v0.1.

Dependencies

• Python: >= 3.10
• isagellm-protocol: >= 0.4.0.0
• isagellm-control-plane: Required for runtime operation.

Installation

Install the gateway and the required control plane package:

pip install isagellm-gateway isagellm-control-plane

Quick Start

Start the Server

Start the gateway server on port 8080:

sagellm-gateway --port 8080

Usage Examples

1. Chat Completion (OpenAI API)

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt2",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

2. List Models

curl http://localhost:8080/v1/models

3. Health Check

curl http://localhost:8080/health

API Documentation

Core Endpoints

• POST /v1/chat/completions: Generate chat completions (supports streaming).
• POST /v1/embeddings: Generate embeddings.
• GET /v1/models: List available models.
• GET /health: Health check and status.

Session Management

• POST /sessions: Create a new session.
• GET /sessions: List all sessions.
• GET /sessions/{session_id}: Get session details.
• DELETE /sessions/{session_id}: Delete a session.

Architecture

┌─────────────────────────────────────────┐
│         sageLLM Gateway                 │
│  • OpenAI-compatible API                │
│  • Session State Management             │
│  • Request Validation & Routing         │
└─────────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────┐
│         Control Plane                   │
│  (managed by isagellm-control-plane)    │
└─────────────────────────────────────────┘

Development Guide

Prerequisites

• Python 3.10+
• Git

Setup

1. Clone the repository:

   git clone [e-mail]:intellistream/sagellm-gateway.git
   cd sagellm-gateway
   

2. Initialize development environment:

# 开发模式（默认）：先装 PyPI，再用本地 editable 覆盖
./quickstart.sh --dev

# 标准模式：仅使用 PyPI 稳定包
./quickstart.sh --standard

# 查看帮助（模式说明与参数）
./quickstart.sh --help

 # Note: You also need the control plane for local testing
 pip install isagellm-control-plane

安装脚本行为说明：

• 安装前会动态清理同前缀历史安装包（默认 isagellm*）
• --dev 模式下本地 editable 覆盖使用 --no-deps，避免重复依赖解析
• 不会创建 venv，复用当前已配置的非-venv Python 环境

Testing

Run the test suite with pytest:

pytest tests/ -v

Linting

Check code quality with ruff:

ruff check .

Actions 账单阻塞时的本地替代 CI

当 GitHub Actions 因 billing/quota 无法启动时，在仓库根目录运行：

bash scripts/local_ci_fallback.sh

该脚本覆盖 ci.yml 与 version-source-guard.yml 的核心检查路径，便于本地复现与验收。

Versioning

Current Version: 0.4.0.11

For version history, see CHANGELOG.md.