sgl-deep-gemm 0.2.0

Introduction

sgl-deep-gemm is a pypi package built from SGLang's customized branch of DeepGemm. Comparing with origina DeepGemm, it supports the following features to better support SGLang:

1. ABI support: with the help of tvm-ffi wrappers, a single wheel can run on different python versions.
2. pypi support: easy installation with pip install sgl-deep-gemm. No need to manually search for wheel links.
3. Fast iteration: add custom kernels and bump versions at no time.

Usage

Build requirements include CUDA 12.9 or newer, a C++20 compiler with std::format support (for example GCC 13), and the elfutils development headers (apt-get install libdw-dev on Debian/Ubuntu). Initialize the CUTLASS and DeepJIT submodules with git submodule update --init --recursive.

To build it locally, run bash build_sgl_deep_gemm.sh, then pip install the wheel generated under dist.

K-grouped FP8 GEMM and its scale-packing helper preserve compact per-group scaling factors by default. Pass use_padded_sf_layout=True to both packing and GEMM when packed scales cover each group's padded K extent. The flag is appended to existing arguments, so existing positional calls retain their behavior.

Run wheel validation with bash sgl_deep_gemm/run_tests.sh. The Mega MoE reference comparisons require DeepEP with ElasticBuffer; Mega Gate and Mega mHC also require TileKernels and TileLang. Install the validation extras with pip install 'sgl-deep-gemm[dev]' (TileLang 0.1.9 and TileKernels 1.0.0). The runner enables NVML-based CUDA discovery to preserve fork compatibility while retaining TVM FFI's DLPack fast path. It runs legacy, lazy-init, and compute-sanitizer checks; --skip-sanitizer is available for ordinary development runs. Distributed MegaMoE runs include separate Hopper numerical accuracy coverage; the sanitizer's automatic discovery covers single-process tests and reports distributed entrypoints separately.

Run bash sgl_deep_gemm/run_tests.sh --release for reduced attention coverage and focused memcheck/synccheck, targeting a release gate under one hour. Selected cases retain their original assertions; other ordinary and distributed tests remain unchanged. The default command keeps the full suite; the table shows release / full attention case counts.

Use Compute Sanitizer 2025.4.1 or newer for Python tests; older releases can retain Python tensors while collecting host backtraces and exhaust GPU memory. Set COMPUTE_SANITIZER to select a separately installed executable. See NVIDIA's release notes.

To release a new set of wheels, please contact SGLang team and run the release workflow under SGLang repo

For each major version release (0.X.Y -> 0.(X+1).0), a new branch should be created (release/v0.(X+1).0) for stability purpose.

For any incoming pull requests, it should be rebased upon dev branch. Any newly added or modified tests should be put under sgl_deep_gemm/tests