logo

PyPI PyPI - Downloads license issue resolution open issues Ask DeepWiki

Blog | Documentation | Roadmap | Join Slack | Weekly Dev Meeting | Slides

News

  • [2026/07] 🔥 SGLang and Miles add day-0 support for Kimi K3 (blog).
  • [2026/07] RadixArk and Google bring full SGLang features to TPUs (blog).
  • [2026/07] Serving GLM5.2 NVFP4 agentic workloads with SGLang: Reaching 500 TPS in two weeks (blog).
  • [2026/06] 🔥 The next generation of speculative decoding: DFlash and Spec V2 (blog).
  • [2026/06] SGLang provides day-0 support for latest open models (Nemotron 3 Ultra, Nemotron 3 Super, Higgs Audio v3 TTS).
  • [2026/04] 🔥 DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles (blog).
  • [2026/02] 🔥 Unlocking 25x Inference Performance with SGLang on NVIDIA GB300 NVL72 (blog).
  • [2026/01] SGLang Diffusion accelerates video and image generation (blog).
More
  • [2025/12] SGLang provides day-0 support for latest open models (MiMo-V2-Flash, Nemotron 3 Nano, Mistral Large 3, LLaDA 2.0 Diffusion LLM, MiniMax M2).
  • [2025/11] SGLang Diffusion accelerates video and image generation (blog).
  • [2025/10] SGLang now runs natively on TPU with the SGLang-Jax backend (blog).
  • [2025/10] PyTorch Conference 2025 SGLang Talk (slide).
  • [2025/10] SGLang x Nvidia SF Meetup on 10/2 (recap).
  • [2025/09] Deploying DeepSeek on GB200 NVL72 with PD and Large Scale EP (Part II): 3.8x Prefill, 4.8x Decode Throughput (blog).
  • [2025/09] SGLang Day 0 Support for DeepSeek-V3.2 with Sparse Attention (blog).
  • [2025/08] SGLang x AMD SF Meetup on 8/22: Hands-on GPU workshop, tech talks by AMD/xAI/SGLang, and networking (Roadmap, Large-scale EP, Highlights, AITER/MoRI, Wave).
  • [2025/08] SGLang provides day-0 support for OpenAI gpt-oss model (instructions)
  • [2025/06] SGLang, the high-performance serving infrastructure powering trillions of tokens daily, has been awarded the third batch of the Open Source AI Grant by a16z (a16z blog).
  • [2025/06] Deploying DeepSeek on GB200 NVL72 with PD and Large Scale EP (Part I): 2.7x Higher Decoding Throughput (blog).
  • [2025/05] Deploying DeepSeek with PD Disaggregation and Large-scale Expert Parallelism on 96 H100 GPUs (blog).
  • [2025/03] Supercharge DeepSeek-R1 Inference on AMD Instinct MI300X (AMD blog)
  • [2025/03] SGLang Joins PyTorch Ecosystem: Efficient LLM Serving Engine (PyTorch blog)
  • [2025/02] Unlock DeepSeek-R1 Inference Performance on AMD Instinct™ MI300X GPU (AMD blog)
  • [2025/01] SGLang provides day one support for DeepSeek V3/R1 models on NVIDIA and AMD GPUs with DeepSeek-specific optimizations. (instructions, AMD blog, 10+ other companies)
  • [2024/12] v0.4 Release: Zero-Overhead Batch Scheduler, Cache-Aware Load Balancer, Faster Structured Outputs (blog).
  • [2024/10] The First SGLang Online Meetup (slides).
  • [2024/09] v0.3 Release: 7x Faster DeepSeek MLA, 1.5x Faster torch.compile, Multi-Image/Video LLaVA-OneVision (blog).
  • [2024/07] v0.2 Release: Faster Llama3 Serving with SGLang Runtime (vs. TensorRT-LLM, vLLM) (blog).
  • [2024/02] SGLang enables 3x faster JSON decoding with compressed finite state machine (blog).
  • [2024/01] SGLang provides up to 5x faster inference with RadixAttention (blog).
  • [2024/01] SGLang powers the serving of the official LLaVA v1.6 release demo (usage).

About

SGLang is a high-performance serving framework for large language models and multimodal models. It is designed to deliver low-latency and high-throughput inference across a wide range of setups, from a single GPU to large distributed clusters. Its core features include:

  • Fast Runtime: Provides efficient serving with RadixAttention for prefix caching, a zero-overhead CPU scheduler, prefill-decode disaggregation, speculative decoding, continuous batching, paged attention, tensor/pipeline/expert/data parallelism, structured outputs, chunked prefill, quantization (FP4/FP8/INT4/AWQ/GPTQ), and multi-LoRA batching.
  • Broad Model Support: Supports a wide range of language models (Llama, Qwen, DeepSeek, Kimi, GLM, GPT, Gemma, Mistral, etc.), embedding models (e5-mistral, gte, mcdse), reward models (Skywork), and diffusion models (WAN, Qwen-Image), with easy extensibility for adding new models. Compatible with most Hugging Face models and OpenAI APIs.
  • Extensive Hardware Support: Runs on NVIDIA GPUs (GB200/B300/H100/A100/Spark/5090), AMD GPUs (MI355/MI300), Intel Xeon CPUs, Google TPUs, Ascend NPUs, and more.
  • Active Community: SGLang is open-source and supported by a vibrant community with widespread industry adoption, powering over 400,000 GPUs worldwide.
  • RL & Post-Training Backbone: SGLang is a proven rollout backend used for training many frontier models, with native RL integrations and adoption by well-known post-training frameworks such as AReaL, Miles, slime, Tunix, verl and more.

Getting Started

Benchmark and Performance

Learn more in the release blogs: v0.2 blog, v0.3 blog, v0.4 blog, Large-scale expert parallelism, GB200 rack-scale parallelism, GB300 long context.

Adoption and Sponsorship

SGLang has been deployed at large scale, generating trillions of tokens in production each day. It is trusted and adopted by a wide range of leading enterprises and institutions, including xAI, NVIDIA, AMD, Intel, LinkedIn, Cursor, Oracle Cloud, Google Cloud, Microsoft Azure, AWS, Atlas Cloud, Voltage Park, Nebius, DataCrunch, Novita, RunPod, InnoMatrix, Modal, MIT, UCLA, the University of Washington, Stanford, UC Berkeley, Tsinghua University, Baseten, Baidu, AntGroup, Alibaba, Tencent, and other major technology organizations. As an open-source LLM inference engine, SGLang has become the de facto industry standard, with deployments running on over 400,000 GPUs worldwide. SGLang is currently hosted under the non-profit open-source organization LMSYS.

logo

Contact Us

For enterprises interested in adopting or deploying SGLang at scale, including technical consulting, sponsorship opportunities, or partnership inquiries, please contact us at sglang@lmsys.org.

Long-term active SGLang contributors are eligible for coding agent sponsorship, such as Cursor, Claude Code, or OpenAI Codex. Email sglang@lmsys.org with your most important commits or pull requests.

Acknowledgment

We learned the design and reused code from the following projects: Guidance, vLLM, LightLLM, FlashInfer, Outlines, and LMQL.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

3 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for kzhou100 from gravatar.com kzhou100 Avatar for lmzheng from gravatar.com lmzheng Avatar for ying1123 from gravatar.com ying1123

License

Apache Software License (Apache License)

Requires

Python >=3.10

Provides Extra

checkpoint-engine runai diffusion ray tracing http2 fastokens test dev all

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

sglang-0.5.17-cp313-cp313-manylinux_2_34_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.34+ x86-64

sglang-0.5.17-cp313-cp313-manylinux_2_34_aarch64.whl (21.8 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.34+ ARM64

sglang-0.5.17-cp312-cp312-manylinux_2_34_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.34+ x86-64

sglang-0.5.17-cp312-cp312-manylinux_2_34_aarch64.whl (21.8 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.34+ ARM64

sglang-0.5.17-cp311-cp311-manylinux_2_34_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.34+ x86-64

sglang-0.5.17-cp311-cp311-manylinux_2_34_aarch64.whl (21.8 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.34+ ARM64

sglang-0.5.17-cp310-cp310-manylinux_2_34_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.34+ x86-64

sglang-0.5.17-cp310-cp310-manylinux_2_34_aarch64.whl (21.8 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.34+ ARM64

File details

Details for the file sglang-0.5.17-cp313-cp313-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for sglang-0.5.17-cp313-cp313-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 b4cec115215aaf355f1ca94122ea0d00318f19f9d9d59846fdb674203efd1f94
MD5 7d4d43869f657138eff1235432b2cd3e
BLAKE2b-256 22c5fb3867372ccb69fd43c8c64bad2c14469ccd9953c4aec50d0f121164fc55

See more details on using hashes here.

File details

Details for the file sglang-0.5.17-cp313-cp313-manylinux_2_34_aarch64.whl.

File metadata

File hashes

Hashes for sglang-0.5.17-cp313-cp313-manylinux_2_34_aarch64.whl
Algorithm Hash digest
SHA256 0239754488a8af898e68df0864ed6d10e9142fede70256d473897578b72e850e
MD5 b6e869b8fb5330de7be5181664e94f50
BLAKE2b-256 dc06e8959ae3d97c0f13cf279631a9350746dbbadddd6841ce5b80753e2ef3c1

See more details on using hashes here.

File details

Details for the file sglang-0.5.17-cp312-cp312-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for sglang-0.5.17-cp312-cp312-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 fb4cba478a5dca5ee146550b39f33add1c4e475d21ee11810b86ed1c58068093
MD5 a8e0cfdb477d829cad58bc80c42309be
BLAKE2b-256 578fe6a8f144d1732717d5c848fa24b0a52013804a21258872cb92bfb87fd30a

See more details on using hashes here.

File details

Details for the file sglang-0.5.17-cp312-cp312-manylinux_2_34_aarch64.whl.

File metadata

File hashes

Hashes for sglang-0.5.17-cp312-cp312-manylinux_2_34_aarch64.whl
Algorithm Hash digest
SHA256 49b3cfa5b998c2ce0962c3da01e8f5e147c610cf7b0d4f8c294c97346c82b202
MD5 eb08df2855751ad1328b7098877b5e21
BLAKE2b-256 94c0abb8e3f5741c572101aae073cc1ca9a521541e7fc3c05660c63c0e468248

See more details on using hashes here.

File details

Details for the file sglang-0.5.17-cp311-cp311-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for sglang-0.5.17-cp311-cp311-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 575e38f5683624ded3b4bb267f637a098e5ed37d13a1d1ff5ea09bd781a04d16
MD5 ff8488f2d6f4da2edfa69b3c982fac2d
BLAKE2b-256 98713b721361cd8bcb76ff9cd51cf22a848cecdfd8cf260c5b015df112854154

See more details on using hashes here.

File details

Details for the file sglang-0.5.17-cp311-cp311-manylinux_2_34_aarch64.whl.

File metadata

File hashes

Hashes for sglang-0.5.17-cp311-cp311-manylinux_2_34_aarch64.whl
Algorithm Hash digest
SHA256 dc67b73978d57eac1a41a41b594c41b0426845a14636546127fa4a99d902f4a0
MD5 1e1ba8f203fd9a937c165ab97556d27b
BLAKE2b-256 d6934748865e3b08ad8380e53671de01d55f3d72df75d4b737e91561533f23fd

See more details on using hashes here.

File details

Details for the file sglang-0.5.17-cp310-cp310-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for sglang-0.5.17-cp310-cp310-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 14f5aed92a0243e0bf5b1e0cc2617295bc21f56687feb2c01a7a8eb768160340
MD5 5e8703e62fb5c4e4f8b58314e4369ba9
BLAKE2b-256 7289d57c253ba8c1a4ce3f87206c674264a250f540c2acbb8f1a1fba7600b49c

See more details on using hashes here.

File details

Details for the file sglang-0.5.17-cp310-cp310-manylinux_2_34_aarch64.whl.

File metadata

File hashes

Hashes for sglang-0.5.17-cp310-cp310-manylinux_2_34_aarch64.whl
Algorithm Hash digest
SHA256 576a689c90e604daf79871233f3a2c9fcbec45cffc9aa8ef6f62308bf2775065
MD5 a1dd045ed397a9188a79f65fd7e5541a
BLAKE2b-256 5833c2c2cb50c7d8ed5d187a5d7a53b36e8b92208a79a67ea05227d114551a00

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

0.5.17

0.5.16

0.5.15.post1

0.5.15

0.5.14

0.5.13.post1

0.5.13

0.5.12.post1

0.5.12

0.5.11

0.5.10.post1

0.5.10

0.5.10rc0 pre-release

0.5.9

0.5.8.post1

0.5.8

0.5.7

0.5.6.post2

0.5.6.post1

0.5.6

0.5.5.post3

0.5.5.post2

0.5.5.post1

0.5.5

0.5.4.post3

0.5.4.post2

0.5.4.post1

0.5.4

0.5.3.post3

0.5.3.post2

0.5.3.post1

0.5.3

0.5.3rc2 pre-release

0.5.3rc0 pre-release

0.5.2

0.5.2rc2 pre-release

0.5.2rc1 pre-release

0.5.2rc0 pre-release

0.5.1.post3

0.5.1.post2

0.5.1.post1

0.5.1

0.5.0rc2 pre-release

0.5.0rc1 pre-release

0.5.0rc0 pre-release

0.4.10.post2

0.4.10.post1

0.4.10

0.4.9.post6

0.4.9.post5

0.4.9.post4

0.4.9.post3

0.4.9.post2

0.4.9.post1

0.4.9

0.4.8.post1

0.4.8

0.4.7.post1

0.4.7

0.4.6.post5

0.4.6.post4

0.4.6.post3

0.4.6.post2

0.4.6.post1

0.4.6

0.4.5.post3

0.4.5.post2

0.4.5.post1

0.4.5

0.4.4.post4

0.4.4.post3

0.4.4.post2

0.4.4.post1

0.4.4

0.4.3.post4

0.4.3.post3

0.4.3.post2

0.4.3.post1

0.4.3

0.4.2.post4

0.4.2.post3

0.4.2.post2

0.4.2.post1

0.4.2

0.4.1.post7

0.4.1.post6

0.4.1.post5

0.4.1.post4

0.4.1.post3

0.4.1.post2

0.4.1.post1

0.4.1

0.4.0.post2

0.4.0.post1

0.4.0

0.3.6.post3

0.3.6.post2

0.3.6.post1

0.3.6

0.3.5.post2

0.3.5.post1

0.3.5

0.3.4.post2

0.3.4.post1

0.3.4

0.3.3.post1

0.3.3

0.3.2

0.3.1.post3

0.3.1.post2

0.3.1.post1

0.3.1

0.3.0

0.2.15

0.2.14.post2

0.2.14.post1

0.2.14

0.2.13

0.2.12

0.2.11

0.2.10

0.2.9.post1

0.2.9

0.2.8

0.2.7

0.2.6

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.26

0.1.25

0.1.24

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.9

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page