logo

PyPI PyPI - Downloads license issue resolution open issues Ask DeepWiki

Blog | Documentation | Roadmap | Join Slack | Weekly Dev Meeting | Slides

News

  • [2026/07] 🔥 SGLang and Miles add day-0 support for Kimi K3 (blog).
  • [2026/07] RadixArk and Google bring full SGLang features to TPUs (blog).
  • [2026/07] Serving GLM5.2 NVFP4 agentic workloads with SGLang: Reaching 500 TPS in two weeks (blog).
  • [2026/06] 🔥 The next generation of speculative decoding: DFlash and Spec V2 (blog).
  • [2026/06] SGLang provides day-0 support for latest open models (Nemotron 3 Ultra, Nemotron 3 Super, Higgs Audio v3 TTS).
  • [2026/04] 🔥 DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles (blog).
  • [2026/02] 🔥 Unlocking 25x Inference Performance with SGLang on NVIDIA GB300 NVL72 (blog).
  • [2026/01] SGLang Diffusion accelerates video and image generation (blog).
More
  • [2025/12] SGLang provides day-0 support for latest open models (MiMo-V2-Flash, Nemotron 3 Nano, Mistral Large 3, LLaDA 2.0 Diffusion LLM, MiniMax M2).
  • [2025/11] SGLang Diffusion accelerates video and image generation (blog).
  • [2025/10] SGLang now runs natively on TPU with the SGLang-Jax backend (blog).
  • [2025/10] PyTorch Conference 2025 SGLang Talk (slide).
  • [2025/10] SGLang x Nvidia SF Meetup on 10/2 (recap).
  • [2025/09] Deploying DeepSeek on GB200 NVL72 with PD and Large Scale EP (Part II): 3.8x Prefill, 4.8x Decode Throughput (blog).
  • [2025/09] SGLang Day 0 Support for DeepSeek-V3.2 with Sparse Attention (blog).
  • [2025/08] SGLang x AMD SF Meetup on 8/22: Hands-on GPU workshop, tech talks by AMD/xAI/SGLang, and networking (Roadmap, Large-scale EP, Highlights, AITER/MoRI, Wave).
  • [2025/08] SGLang provides day-0 support for OpenAI gpt-oss model (instructions)
  • [2025/06] SGLang, the high-performance serving infrastructure powering trillions of tokens daily, has been awarded the third batch of the Open Source AI Grant by a16z (a16z blog).
  • [2025/06] Deploying DeepSeek on GB200 NVL72 with PD and Large Scale EP (Part I): 2.7x Higher Decoding Throughput (blog).
  • [2025/05] Deploying DeepSeek with PD Disaggregation and Large-scale Expert Parallelism on 96 H100 GPUs (blog).
  • [2025/03] Supercharge DeepSeek-R1 Inference on AMD Instinct MI300X (AMD blog)
  • [2025/03] SGLang Joins PyTorch Ecosystem: Efficient LLM Serving Engine (PyTorch blog)
  • [2025/02] Unlock DeepSeek-R1 Inference Performance on AMD Instinct™ MI300X GPU (AMD blog)
  • [2025/01] SGLang provides day one support for DeepSeek V3/R1 models on NVIDIA and AMD GPUs with DeepSeek-specific optimizations. (instructions, AMD blog, 10+ other companies)
  • [2024/12] v0.4 Release: Zero-Overhead Batch Scheduler, Cache-Aware Load Balancer, Faster Structured Outputs (blog).
  • [2024/10] The First SGLang Online Meetup (slides).
  • [2024/09] v0.3 Release: 7x Faster DeepSeek MLA, 1.5x Faster torch.compile, Multi-Image/Video LLaVA-OneVision (blog).
  • [2024/07] v0.2 Release: Faster Llama3 Serving with SGLang Runtime (vs. TensorRT-LLM, vLLM) (blog).
  • [2024/02] SGLang enables 3x faster JSON decoding with compressed finite state machine (blog).
  • [2024/01] SGLang provides up to 5x faster inference with RadixAttention (blog).
  • [2024/01] SGLang powers the serving of the official LLaVA v1.6 release demo (usage).

About

SGLang is a high-performance serving framework for large language models and multimodal models. It is designed to deliver low-latency and high-throughput inference across a wide range of setups, from a single GPU to large distributed clusters. Its core features include:

  • Fast Runtime: Provides efficient serving with RadixAttention for prefix caching, a zero-overhead CPU scheduler, prefill-decode disaggregation, speculative decoding, continuous batching, paged attention, tensor/pipeline/expert/data parallelism, structured outputs, chunked prefill, quantization (FP4/FP8/INT4/AWQ/GPTQ), and multi-LoRA batching.
  • Broad Model Support: Supports a wide range of language models (Llama, Qwen, DeepSeek, Kimi, GLM, GPT, Gemma, Mistral, etc.), embedding models (e5-mistral, gte, mcdse), reward models (Skywork), and diffusion models (WAN, Qwen-Image), with easy extensibility for adding new models. Compatible with most Hugging Face models and OpenAI APIs.
  • Extensive Hardware Support: Runs on NVIDIA GPUs (GB200/B300/H100/A100/Spark/5090), AMD GPUs (MI355/MI300), Intel Xeon CPUs, Google TPUs, Ascend NPUs, and more.
  • Active Community: SGLang is open-source and supported by a vibrant community with widespread industry adoption, powering over 400,000 GPUs worldwide.
  • RL & Post-Training Backbone: SGLang is a proven rollout backend used for training many frontier models, with native RL integrations and adoption by well-known post-training frameworks such as AReaL, Miles, slime, Tunix, verl and more.

Getting Started

Benchmark and Performance

Learn more in the release blogs: v0.2 blog, v0.3 blog, v0.4 blog, Large-scale expert parallelism, GB200 rack-scale parallelism, GB300 long context.

Adoption and Sponsorship

SGLang has been deployed at large scale, generating trillions of tokens in production each day. It is trusted and adopted by a wide range of leading enterprises and institutions, including xAI, NVIDIA, AMD, Intel, LinkedIn, Cursor, Oracle Cloud, Google Cloud, Microsoft Azure, AWS, Atlas Cloud, Voltage Park, Nebius, DataCrunch, Novita, RunPod, InnoMatrix, Modal, MIT, UCLA, the University of Washington, Stanford, UC Berkeley, Tsinghua University, Baseten, Baidu, AntGroup, Alibaba, Tencent, and other major technology organizations. As an open-source LLM inference engine, SGLang has become the de facto industry standard, with deployments running on over 400,000 GPUs worldwide. SGLang is currently hosted under the non-profit open-source organization LMSYS.

logo

Contact Us

For enterprises interested in adopting or deploying SGLang at scale, including technical consulting, sponsorship opportunities, or partnership inquiries, please contact us at sglang@lmsys.org.

Long-term active SGLang contributors are eligible for coding agent sponsorship, such as Cursor, Claude Code, or OpenAI Codex. Email sglang@lmsys.org with your most important commits or pull requests.

Acknowledgment

We learned the design and reused code from the following projects: Guidance, vLLM, LightLLM, FlashInfer, Outlines, and LMQL.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

3 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for kzhou100 from gravatar.com kzhou100 Avatar for lmzheng from gravatar.com lmzheng Avatar for ying1123 from gravatar.com ying1123

License

Apache Software License (Apache License)

Requires

Python >=3.10

Provides Extra

checkpoint-engine runai diffusion diffusion-qvg ray tracing http2 fastokens test dev all

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

sglang-0.5.18-cp313-cp313-manylinux_2_34_x86_64.whl (24.0 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.34+ x86-64

sglang-0.5.18-cp313-cp313-manylinux_2_34_aarch64.whl (23.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.34+ ARM64

sglang-0.5.18-cp312-cp312-manylinux_2_34_x86_64.whl (24.0 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.34+ x86-64

sglang-0.5.18-cp312-cp312-manylinux_2_34_aarch64.whl (23.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.34+ ARM64

sglang-0.5.18-cp311-cp311-manylinux_2_34_x86_64.whl (24.0 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.34+ x86-64

sglang-0.5.18-cp311-cp311-manylinux_2_34_aarch64.whl (23.4 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.34+ ARM64

sglang-0.5.18-cp310-cp310-manylinux_2_34_x86_64.whl (24.0 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.34+ x86-64

sglang-0.5.18-cp310-cp310-manylinux_2_34_aarch64.whl (23.4 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.34+ ARM64

File details

Details for the file sglang-0.5.18-cp313-cp313-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for sglang-0.5.18-cp313-cp313-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 8a4fae5995751f32278f9466da6cb77d6dace1ae92b89a30ff8fdcf0c621c324
MD5 4652d2dec59d76f2ebad15e9e117b9dd
BLAKE2b-256 f0d5aa019659664eec0cb160ea19e155cadf6b88df5c911ba83ef0c5e241aa75

See more details on using hashes here.

File details

Details for the file sglang-0.5.18-cp313-cp313-manylinux_2_34_aarch64.whl.

File metadata

File hashes

Hashes for sglang-0.5.18-cp313-cp313-manylinux_2_34_aarch64.whl
Algorithm Hash digest
SHA256 acbb5bc107e42f6b9f364be12a248be4ec993085a8c6963df451def2437c75f3
MD5 fd0fda9b11f6e403dbbda75de13840ee
BLAKE2b-256 14636261496319b4fa23a5d4d4f5a23087071a5f1e1bb75ef84643efa0033a96

See more details on using hashes here.

File details

Details for the file sglang-0.5.18-cp312-cp312-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for sglang-0.5.18-cp312-cp312-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 fba7bb31a5de4c014945cbc50e9357c9253cc965a423736c324b2aaccd2270dc
MD5 14cebb52d1237c9dd05e8d686c1d3f17
BLAKE2b-256 48028c15afa2bb3f4d060d4ff7a8cd66d35bfa233f94b41f02a2d59b439dbfde

See more details on using hashes here.

File details

Details for the file sglang-0.5.18-cp312-cp312-manylinux_2_34_aarch64.whl.

File metadata

File hashes

Hashes for sglang-0.5.18-cp312-cp312-manylinux_2_34_aarch64.whl
Algorithm Hash digest
SHA256 279621e464e56c597150e542321350ea56e000e192d52fcffbd4cbe9ea80ca7f
MD5 12c49a012bacc835348ac982e99c3b77
BLAKE2b-256 3d27ac8038a62e9f9f569b599e25366b6c53050f88a3c49b57eff54573f01489

See more details on using hashes here.

File details

Details for the file sglang-0.5.18-cp311-cp311-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for sglang-0.5.18-cp311-cp311-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 e97b585dd2d5db2802599feba47c84cce972f5e1231a6816fef996200207b5cf
MD5 1f302cced3b952f7094018788d2d7363
BLAKE2b-256 d9feb1f48d474b18a0d4035911ea8eb20058afef1fc7342c3243b7123aafca36

See more details on using hashes here.

File details

Details for the file sglang-0.5.18-cp311-cp311-manylinux_2_34_aarch64.whl.

File metadata

File hashes

Hashes for sglang-0.5.18-cp311-cp311-manylinux_2_34_aarch64.whl
Algorithm Hash digest
SHA256 fae5e92a9f3b880974ac731eb570e3d164886291807473bd95d2458f5b28957e
MD5 c7858c627ec1cbef8a02ced830a564c0
BLAKE2b-256 421d1eeefb8b556151ea5159b17e5218b3892d4456e0c780d5769d418160a383

See more details on using hashes here.

File details

Details for the file sglang-0.5.18-cp310-cp310-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for sglang-0.5.18-cp310-cp310-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 8a7e596914bab0fec0ebff16c430dfd04077fb673c81008c8eb6aa584e3da4e2
MD5 7992a9d541c9d1a690dfb86a7e0d10af
BLAKE2b-256 16e30951e72d408097979a4a5946573d9f628568f9add9337aa07afaf0434ee0

See more details on using hashes here.

File details

Details for the file sglang-0.5.18-cp310-cp310-manylinux_2_34_aarch64.whl.

File metadata

File hashes

Hashes for sglang-0.5.18-cp310-cp310-manylinux_2_34_aarch64.whl
Algorithm Hash digest
SHA256 f8384cec302537fbe31aa271e02ca6a318fa64547a7e10a0cd4c03f7e60f2dce
MD5 c86aa5f77478b00fc065c236a923c55f
BLAKE2b-256 449245b34a39fb0d455d79d184aff1dd6421b5eebd3d85b9d013c973bcc6c97d

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

0.5.18 This release

8 files

0.5.17

8 files

0.5.16

8 files

0.5.15.post1

8 files

0.5.15

8 files

0.5.14

8 files

0.5.13.post1

8 files

0.5.13

8 files

0.5.12.post1

8 files

0.5.12

8 files

0.5.11

8 files

0.5.10.post1

2 files

0.5.10

2 files

Pre-release

0.5.10rc0

2 files

0.5.9

2 files

0.5.8.post1

2 files

0.5.8

2 files

0.5.7

2 files

0.5.6.post2

2 files

0.5.6.post1

2 files

0.5.6

2 files

0.5.5.post3

2 files

0.5.5.post2

2 files

0.5.5.post1

2 files

0.5.5

2 files

0.5.4.post3

2 files

0.5.4.post2

2 files

0.5.4.post1

2 files

0.5.4

2 files

0.5.3.post3

2 files

0.5.3.post2

2 files

0.5.3.post1

2 files

0.5.3

2 files

Pre-release

0.5.3rc2

2 files

Pre-release

0.5.3rc0

2 files

0.5.2

2 files

Pre-release

0.5.2rc2

2 files

Pre-release

0.5.2rc1

2 files

Pre-release

0.5.2rc0

2 files

0.5.1.post3

2 files

0.5.1.post2

2 files

0.5.1.post1

2 files

0.5.1

2 files

Pre-release

0.5.0rc2

2 files

Pre-release

0.5.0rc1

2 files

Pre-release

0.5.0rc0

2 files

0.4.10.post2

2 files

0.4.10.post1

2 files

0.4.10

2 files

0.4.9.post6

2 files

0.4.9.post5

2 files

0.4.9.post4

2 files

0.4.9.post3

2 files

0.4.9.post2

2 files

0.4.9.post1

2 files

0.4.9

2 files

0.4.8.post1

2 files

0.4.8

2 files

0.4.7.post1

2 files

0.4.7

2 files

0.4.6.post5

2 files

0.4.6.post4

2 files

0.4.6.post3

2 files

0.4.6.post2

2 files

0.4.6.post1

2 files

0.4.6

2 files

0.4.5.post3

2 files

0.4.5.post2

2 files

0.4.5.post1

2 files

0.4.5

2 files

0.4.4.post4

2 files

0.4.4.post3

2 files

0.4.4.post2

2 files

0.4.4.post1

2 files

0.4.4

2 files

0.4.3.post4

2 files

0.4.3.post3

2 files

0.4.3.post2

2 files

0.4.3.post1

2 files

0.4.3

2 files

0.4.2.post4

2 files

0.4.2.post3

2 files

0.4.2.post2

2 files

0.4.2.post1

2 files

0.4.2

2 files

0.4.1.post7

2 files

0.4.1.post6

2 files

0.4.1.post5

2 files

0.4.1.post4

2 files

0.4.1.post3

2 files

0.4.1.post2

2 files

0.4.1.post1

2 files

0.4.1

2 files

0.4.0.post2

2 files

0.4.0.post1

2 files

0.4.0

2 files

0.3.6.post3

2 files

0.3.6.post2

2 files

0.3.6.post1

2 files

0.3.6

2 files

0.3.5.post2

2 files

0.3.5.post1

2 files

0.3.5

2 files

0.3.4.post2

2 files

0.3.4.post1

2 files

0.3.4

2 files

0.3.3.post1

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1.post3

2 files

0.3.1.post2

2 files

0.3.1.post1

2 files

0.3.1

2 files

0.3.0

2 files

0.2.15

2 files

0.2.14.post2

2 files

0.2.14.post1

2 files

0.2.14

2 files

0.2.13

2 files

0.2.12

2 files

0.2.11

2 files

0.2.10

2 files

0.2.9.post1

2 files

0.2.9

2 files

0.2.8

2 files

0.2.7

2 files

0.2.6

2 files

0.2.5

2 files

0.2.4

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.26

2 files

0.1.25

2 files

0.1.24

2 files

0.1.22

2 files

0.1.21

2 files

0.1.20

2 files

0.1.19

2 files

0.1.18

2 files

0.1.17

2 files

0.1.16

2 files

0.1.15

2 files

0.1.14

2 files

0.1.13

2 files

0.1.12

2 files

0.1.11

2 files

0.1.10

2 files

0.1.9

2 files

0.1.7

2 files

0.1.6

2 files

0.1.5

2 files

0.1.4

2 files

0.1.3

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page