Distributed inference for diffusion transformers
Documentation site · Quick start · Supported models · Discussions · PapersxDiT
xDiT runs image, video, and audiovisual diffusion models on one or more devices.
The xdit CLI selects a model, configures parallel execution, and saves generated
outputs and measurements. Features vary by model and hardware; start with the
supported-model and capability tables.
Quick start
Install
Install PyTorch for your device, then install xDiT:
pip install xfuser
# Optional: Cache-DiT caching and TorchAO quantization
pip install "xfuser[cache-dit,torchao]"
The base package includes DistVAE and Yunchang. Some models require newer Diffusers or Transformers than the package minimum. See the installation guide for source builds, optional packages, and CUDA, ROCm, and Ascend containers.
Run a model
Choose a model from the supported-model list. This example uses FLUX.1-dev on one GPU with enough memory for its weights:
xdit --model FLUX.1-dev --prompt "A cat running in a garden"
Accept the checkpoint's license and authenticate with Hugging Face if required. The first run downloads the weights. To distribute attention across eight GPUs:
xdit --model FLUX.1-dev --ulysses_degree 8 \
--prompt "A cat running in a garden"
The launcher infers the process count from the parallel degrees. See launching, legacy pipeline examples, and the CLI reference for other workflows and options.
Choose capabilities
| Goal | Guide |
|---|---|
| Scale inference across GPUs | Ulysses and Ring, hybrid layouts |
| Fit models and high-resolution outputs in memory | Sharding and offload, parallel VAE |
| Tune kernels, compute, and communication | Attention, quantization, communication |
| Reuse denoising work | Caching |
| Adapt, measure, compile, and profile | LoRA, benchmarking |
Check the model capability matrix before combining features. Quantization, caching, and reduced-precision communication can change output quality; compare results on your workload.
Integrations and resources
- xDiT ComfyUI: model loading and generation through ComfyUI.
- Legacy model examples.
- External model integrations: StepVideo and Mochi use separate launchers.
- Container builds and workload configurations: CUDA, Instinct, and RDNA 4 environments.
- Community resources, including Google Cloud serving and ROCm recipes.
Contribute
Start with adding a model and framework design. Use discussions for questions and issues for reproducible problems.
📝 Cite Us
xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
@article{fang2024xdit,
title={xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism},
author={Fang, Jiarui and Pan, Jinzhe and Sun, Xibo and Li, Aoyu and Wang, Jiannan},
journal={arXiv preprint arXiv:2411.01738},
year={2024}
}
PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference
@inproceedings{
fang2025pipefusion,
title={PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference},
author={Jiarui Fang and Jinzhe Pan and Aoyu Li and Xibo Sun and WANG Jiannan},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=5xwyxupsLL}
}
USP: A Unified Sequence Parallelism Approach for Long Context Generative AI
@article{fang2024unified,
title={A Unified Sequence Parallelism Approach for Long Context Generative AI},
author={Fang, Jiarui and Zhao, Shangchun},
journal={arXiv preprint arXiv:2405.07719},
year={2024}
}
Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Study
@article{sun2024unveiling,
title={Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Study},
author={Sun, Xibo and Fang, Jiarui and Li, Aoyu and Pan, Jinzhe},
journal={arXiv preprint arXiv:2411.13588},
year={2024}
}
Metadata
Release files for xfuser 0.8.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| xfuser-0.8.0.tar.gz | 1.4 MB | Without provenance | Details for xfuser-0.8.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| xfuser-0.8.0-py3-none-any.whl | Python 3 | none | any | Details for xfuser-0.8.0-py3-none-any.whl |
Total release size: 3.2 MB
Release files / xfuser-0.8.0.tar.gz
| Download URL | xfuser-0.8.0.tar.gz |
|---|---|
| Size | 1.4 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d2d4773d675903bf1cb9b251610f223ad2be257c014c0096c83ff06a33c6d81a
|
|
BLAKE2b-256 checksum How to use checksums |
e13b5bd7ca13ea348992a23216dbcb133c8b7f64dbd07c281dcdefd1daf3f1ab
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.25
|
Release files / xfuser-0.8.0-py3-none-any.whl
| Download URL | xfuser-0.8.0-py3-none-any.whl |
|---|---|
| Size | 1.8 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
153ecae2af96835e791224e1153646bec1231491c68d07044f081f2e8097cad8
|
|
BLAKE2b-256 checksum How to use checksums |
b16f0c19c52e6c8b6dfc818ed44afc91d76cbf528ca4e1a4d8e087c49fa7449b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.25
|
Security for xfuser 0.8.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.