sgl-kernel 0.0.3.post7

Kernel Library for SGLang

SGL Kernel

Kernel Library for SGLang

Installation

For CUDA 11.8:

pip3 install sgl-kernel -i https://docs.sglang.ai/whl/cu118

For CUDA 12.1 or CUDA 12.4:

pip3 install sgl-kernel

Developer Guide

Development Environment Setup

Use Docker to set up the development environment. See Docker setup guide.

Create and enter development container:

docker run -itd --shm-size 32g --gpus all -v $HOME/.cache:/root/.cache --ipc=host --name sglang_zhyncs lmsysorg/sglang:dev /bin/zsh
docker exec -it sglang_zhyncs /bin/zsh

Project Structure

Dependencies

Third-party libraries:

• CCCL
• CUTLASS
• FlashInfer
• TurboMind

Kernel Development

Steps to add a new kernel:

1. Implement in src/sgl-kernel/csrc/
2. Expose interface in src/sgl-kernel/include/sgl_kernels_ops.h
3. Create torch extension in src/sgl-kernel/torch_extension.cc
4. Create Python wrapper in src/sgl-kernel/ops/__init__.py
5. Expose Python interface in src/sgl-kernel/__init__.py
6. Update setup.py to include new CUDA source

Build & Install

Development build:

make build

Note:

The sgl-kernel is rapidly evolving. If you experience a compilation failure, try using make rebuild.

Testing & Benchmarking

1. Add pytest tests in tests/
2. Add benchmarks using triton benchmark in benchmark/
3. Run test suite

Release new version

Update version in pyproject.toml and version.py