sglang-router 0.0.4

SGLang router is a standalone module implemented in Rust to achieve data parallelism across SGLang instances.

SGLang Router (Experimental)

SGLang router is a standalone module implemented in Rust to achieve data parallelism across SGLang instances.

Prerequisites

• Rust and Cargo installed

# Install rustup (Rust installer and version manager)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Follow the installation prompts, then reload your shell
source $HOME/.cargo/env

# Verify installation
rustc --version
cargo --version

• Python with pip installed

Build Process

1. Build Rust Project

cargo build

2. Build Python Binding

Option A: Build and Install Wheel

1. Build the wheel package:

pip install setuptools-rust wheel build
python -m build

2. Install the generated wheel:

pip install <path-to-wheel>

Option B: Development Mode

For development purposes, you can install the package in editable mode:

pip install -e .

Note: When modifying Rust code, you must rebuild the wheel for changes to take effect.

CI/CD Setup

The continuous integration pipeline consists of three main steps:

1. Build Wheels

• Uses cibuildwheel to create manylinux x86_64 packages
• Compatible with major Linux distributions (Ubuntu, CentOS, etc.)
• Additional configurations can be added to support other OS/architectures
• Reference: cibuildwheel documentation

2. Build Source Distribution

• Creates a source distribution containing the raw, unbuilt code
• Enables pip to build the package from source when prebuilt wheels are unavailable

3. Publish to PyPI

• Uploads both wheels and source distribution to PyPI

The CI configuration is based on the tiktoken workflow.