Efficient, Flexible and Portable Structured Generation
News
- [2025/02] XGrammar has been officially integrated into Modular's MAX
- [2025/01] XGrammar has been officially integrated into TensorRT-LLM.
- [2024/12] XGrammar has been officially integrated into vLLM.
- [2024/12] We presented research talks on XGrammar at CMU, UC Berkeley, MIT, THU, SJTU, Ant Group, LMSys, Qingke AI, Camel AI. The slides can be found here.
- [2024/11] XGrammar has been officially integrated into SGLang.
- [2024/11] XGrammar has been officially integrated into MLC-LLM.
- [2024/11] We officially released XGrammar v0.1.0!
Overview
XGrammar is an open-source library for efficient, flexible, and portable structured generation.
It leverages constrained decoding to ensure 100% structural correctness of the output. It supports general context-free grammar to enable a broad range of structures, including JSON, regex, custom context-free grammar, etc.
XGrammar uses careful optimizations to achieve extremely low overhead in structured generation. It has achieved near-zero overhead in JSON generation, making it one of the fastest structured generation engines available.
XGrammar features universal deployment. It supports:
- Platforms: Linux, macOS, Windows
- Hardware: CPU, NVIDIA GPU, AMD GPU, Apple Silicon, TPU, etc.
- Languages: Python, C++, and JavaScript APIs
- Models: Qwen, Llama, DeepSeek, Phi, Gemma, etc.
XGrammar is very easy to integrate with LLM inference engines. It is the default structured generation backend for most LLM inference engines, including vLLM, SGLang, TensorRT-LLM, and MLC-LLM, as well as many other companies. You can also try out their structured generation modes!
Get Started
Install XGrammar:
pip install xgrammar
Import XGrammar:
import xgrammar as xgr
Please visit our documentation to get started with XGrammar.
Adoption
XGrammar has been adopted by many projects and companies, including but not limited to:
Citation
If you find XGrammar useful in your research, please consider citing our paper:
@article{dong2024xgrammar,
title={Xgrammar: Flexible and efficient structured generation engine for large language models},
author={Dong, Yixin and Ruan, Charlie F and Cai, Yaxing and Lai, Ruihang and Xu, Ziyi and Zhao, Yilong and Chen, Tianqi},
journal={Proceedings of Machine Learning and Systems 7},
year={2024}
}
Metadata
Release files for b10-xgrammar 0.1.22
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| b10_xgrammar-0.1.22-cp312-cp312-manylinux_2_38_x86_64.whl | CPython 3.12 | CPython 3.12 | Linux glibc 2.38+ x86-64 | Details for b10_xgrammar-0.1.22-cp312-cp312-manylinux_2_38_x86_64.whl |
Release files / b10_xgrammar-0.1.22-cp312-cp312-manylinux_2_38_x86_64.whl
| Download URL | b10_xgrammar-0.1.22-cp312-cp312-manylinux_2_38_x86_64.whl |
|---|---|
| Size | 7.5 MB |
| Tags | CPython 3.12 Linux glibc 2.38+ x86-64 |
|
SHA-256 checksum How to use checksums |
7bdac22cb15e95a6023b152f05a8f193e2e6cda78e4d3f77e0f6452c40080a3b
|
|
BLAKE2b-256 checksum How to use checksums |
d3ecbd47cf2583c9ee28fcffda77640a93365fe0b6997d3b350988d95a96e54a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.6
|
Security for b10-xgrammar 0.1.22
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.