Skip to main content

⚡️🎉A PyTorch-native Inference Engine with Cache,
Parallelism, Quantization and CPU Offload for DiTs
Featured|HelloGitHub

**🤗Why Cache-DiT❓❓**Cache-DiT is built on top of the 🤗Diffusers library and now supports nearly ALL DiTs from Diffusers. It provides hybrid cache acceleration (DBCache, TaylorSeer, SCM, etc.) and comprehensive parallelism optimizations, including Context Parallelism, Tensor Parallelism, hybrid 2D or 3D parallelism, and dedicated extra parallelism support for Text Encoder, VAE, and ControlNet.

Cache-DiT is compatible with compilation, offload, and quantization, fully integrates with SGLang Diffusion, vLLM-Omni, TensorRT-LLM, xDiT, and runs natively on NVIDIA GPUs, Huawei Ascend NPUs and AMD GPUs. Cache-DiT is fast, easy to use, and flexible for various DiTs (online docs at 📘cache-dit.io).

⚡️9x speedup by Cache-DiT with Cache, Context Parallelism and Compilation

🚀Quick Start: Cache, Parallelism and Quantization

pip install -U cache-dit # PyPI, stable release.
pip install git+https://github.com/vipshop/cache-dit.git # latest.
>>> import cache_dit
>>> from diffusers import DiffusionPipeline
>>> pipe = DiffusionPipeline.from_pretrained(...).to("cuda")
>>> cache_dit.enable_cache(pipe) # Cache Acceleration with One-line code.
>>> from cache_dit import DBCacheConfig, ParallelismConfig
>>> cache_dit.enable_cache( # Or, Hybrid Cache Acceleration + Parallelism.
...   pipe, cache_config=DBCacheConfig(), # w/ default
...   parallelism_config=ParallelismConfig(ulysses_size=2))
>>> from cache_dit import DBCacheConfig, ParallelismConfig, QuantizeConfig
>>> cache_dit.enable_cache( # Or, Hybrid Cache + Parallelism + Quantization.
...   pipe, cache_config=DBCacheConfig(), # w/ default
...   parallelism_config=ParallelismConfig(ulysses_size=2),
...   quantize_config=QuantizeConfig(quant_type=...))
>>> output = pipe(...) # Then, just call the pipe as normal.

🚀Quick Start: SVDQuant (W4A4) PTQ/DQ workflow

# Required: CUDA >=13.0, PyTorch >=2.11 (a CUDA 13 build), GLIBC 2.38.
pip install -U cache-dit --extra-index-url https://vipshop.github.io/cache-dit/whl/cu13/ 
# Latest, from source: one flag builds every CUDA extension (SVDQuant, comm, attn).
git submodule update --init --recursive --force # the SVDQuant sources need spdlog
CACHE_DIT_BUILD_CUDA=1 pip install -e ".[quantization]" # default, all archs.
>>> from cache_dit import QuantizeConfig
>>> pipe = DiffusionPipeline.from_pretrained(...).to("cuda")
>>> # ✅ PTQ: "...{dtype}_r{rank}", required `calibrate_fn`.
>>> def calibrate_fn(**_: object) -> None:
...  for prompt in calibration_prompts: _ = pipe(...)
>>> pipe.transformer = cache_dit.quantize(
...  pipe.transformer, quant_config=QuantizeConfig(
...  quant_type="svdq_nvfp4_r32", calibrate_fn=calibrate_fn,
...  serialize_to="path"))
>>> pipe.transformer = cache_dit.load(pipe.transformer, "path")
>>> # ✅ DQ: "...{dtype}_r{rank}_dq", NO calibration need.
>>> pipe.transformer = cache_dit.quantize(
...   pipe.transformer, quant_config=QuantizeConfig(
...   quant_type="svdq_{int4|nvfp4}_r{32|64|128|256|...}_dq",
...   svdq_kwargs={"smooth_strategy": "few_shot"})) 
>>> output = pipe(...) # Then, just call the pipe as normal.

🚀Quick Start: Bucket-style Layerwise CPU Offload

>>> import cache_dit
>>> cache_dit.layerwise_offload(
...   pipe, # nn.Module: pipe, transformer, text_encoder, etc.
...   onload_device="cuda",
...   offload_device="cpu",
...   async_transfer=True,
...   transfer_buckets=4,
...   persistent_buckets=64,
...   persistent_bins=8,
...   prefetch_limit=True,
...   max_copy_streams=4,
...   max_inflight_prefetch_bytes="8gib")
>>> output = pipe(...) # Then, just call the pipe as normal.

🚪Supported DiT Models

Cache-DiT supports 60+ DiT pipeline families (150+ Variants) from 🤗Diffusers, covering the vast majority of DiT-based pipelines. For full support matrix and detailed usage, please refer to our documentation at 📘cache-dit.io.

Modality Pipeline Series Model Sources Variants C/P/Q/OF
Video Pika H3 Flex (T2VA) 🔥 🤗diffusers@0.43.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video Pika H3 Flex (FL2VA) 🔥 🤗diffusers@0.43.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video Pika H3 Flex (REF2VA) 🔥 🤗diffusers@0.43.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video SANA-Video 2B (T2V) 🔥 🤗diffusers@0.42.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video SANA-Video 2B (I2V) 🔥 🤗diffusers@0.42.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video SANA-WM (I2V) 🔥 🤗diffusers@0.42.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image FLUX.3 Klein 🔥 🤗diffusers@0.42.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video Wan-Animate-2 🔥 🤗diffusers@0.42.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Image Bria-Fibo 2.0 🔥 🤗diffusers@0.42.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Video Kandinsky 6.0 (T2VA) 🔥 🤗diffusers@0.42.0, 🤗HF 1 ✔️/✔️/✔️/✖️
Audio Stable Audio 3 (T2A) 🔥 🤗diffusers@0.42.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Audio LongCat-AudioDiT (TTS) 🔥 🤗diffusers@0.42.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Image Qwen-Image-2.1 🔥 🤗diffusers@0.41.0, 🤗HF 3 ✔️/✔️/✔️/✔️
Video LTX2.5 (DFR/T2V/I2V) 🔥 🤗diffusers@0.41.0, 🤗HF 6 ✔️/✔️/✔️/✔️
Video Echo (JoyAI-Echo) 🔥 🤗diffusers@0.41.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video MiniMax-H3 (T2VA) 🔥 🤗diffusers@0.40.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video MiniMax-H3 (FL2VA) 🔥 🤗diffusers@0.40.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video MiniMax-H3 (Ref2VA) 🔥 🤗diffusers@0.40.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image MiniMax-H3-Image (T2I) 🔥 🤗diffusers@0.42.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image MiniMax-H3-Image (Edit) 🔥 🤗diffusers@0.42.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image P-H3-Image (T2I) 🔥 🤗diffusers@0.43.0, 🤗HF 1 ✔️/✔️/✖️/✔️
Image P-H3-Image (Edit) 🔥 🤗diffusers@0.43.0, 🤗HF 1 ✔️/✔️/✖️/✔️
Image Joy-Image 🤗diffusers@0.39.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image Krea-2 🤗diffusers@0.39.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image Ideogram 4 🔥 🤗diffusers@0.39.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image Anima 🤗diffusers@0.39.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video Any-Flow 🤗diffusers@0.39.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video Any-Flow-FAR 🤗diffusers@0.39.0, 🤗HF 1 ✖️/✖️/✔️/✔️
Video Cosmos3 (Omni) 🔥 🤗diffusers@0.39.0, 🤗HF 2 ✔️/✖️/✔️/✔️
Video Motif-Video (T2V/I2V) 🔥 🤗diffusers@0.39.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Image ERNIE-Image 🔥 🤗diffusers@0.38.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image NucleusMoE-Image 🔥 🤗diffusers@0.38.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image FLUX.2 Klein 🔥 🤗diffusers@0.37.0, 🤗HF 3 ✔️/✔️/✔️/✔️
Image GLM-Image 🔥 🤗diffusers@0.37.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image LongCat-Image 🔥 🤗diffusers@0.37.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Video LTX2 🤗diffusers@0.37.0, 🤗HF 6 ✔️/✔️/✔️/✔️
Video Helios 🔥 🤗diffusers@0.37.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Image FLUX.2 🔥 🤗diffusers@0.36.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image Z-Image 🔥 🤗diffusers@0.36.0, 🤗HF 6 ✔️/✔️/✔️/✔️
Image Bria-Fibo 🤗diffusers@0.36.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Image Ovis-Image 🤗diffusers@0.36.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image Hunyuan-Image 🔥 🤗diffusers@0.36.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Image Bria 🤗diffusers@0.36.0, 🤗HF 1 ✔️/✖️/✔️/✔️
Image PRX / PRXPixel 🔥 🤗diffusers@0.36.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Image FireRed-Image-Edit 🤗diffusers@0.36.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Video HunyuanVideo 1.5 🔥 🤗diffusers@0.36.0, 🤗HF 2 ✔️/✖️/✔️/✔️
Video ChronoEdit 🤗diffusers@0.36.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video Kandinsky5 (T2V/I2V) 🤗diffusers@0.36.0, 🤗HF 4 ✔️/✔️/✔️/✔️
Other LucyEdit 🤗diffusers@0.36.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image Qwen-Image 🔥 🤗diffusers@0.35.0, 🤗HF 9+ ✔️/✔️/✔️/✔️
Video Wan 2.2 (T2V/I2V) 🔥 🤗diffusers@0.35.0, 🤗HF 3 ✔️/✔️/✔️/✔️
Video SkyReelsV2 🤗diffusers@0.35.0, 🤗HF 5 ✔️/✔️/✔️/✔️
Image HiDream 🤗diffusers@0.34.0, 🤗HF 1 ✔️/✖️/✔️/✔️
Image Chroma 🤗diffusers@0.34.0, 🤗HF 3 ✔️/✔️/✔️/✔️
Image VisualCloze 🤗diffusers@0.34.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Video Cosmos 🤗diffusers@0.34.0, 🤗HF 7+ ✔️/✖️/✔️/✔️
Video Wan-VACE 🔥 🤗diffusers@0.34.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image CogView4 🤗diffusers@0.33.0, 🤗HF 2 ✔️/✔️/✔️/✔️
Image OmniGen 🤗diffusers@0.33.0, 🤗HF 1 ✔️/✖️/✔️/✔️
Image Lumina 2 🤗diffusers@0.33.0, 🤗HF 2 ✔️/✖️/✔️/✔️
Video Wan 2.1 (T2V/I2V) 🔥 🤗diffusers@0.33.0, 🤗HF 3 ✔️/✔️/✔️/✔️
Video EasyAnimate 🤗diffusers@0.33.0, 🤗HF 3 ✔️/✖️/✔️/✔️
Video ConsisID 🤗diffusers@0.33.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image Sana (image) 🤗diffusers@0.32.0, 🤗HF 4 ✔️/✖️/✔️/✔️
Video HunyuanVideo 🤗diffusers@0.32.0, 🤗HF 4 ✔️/✔️/✔️/✔️
Video Mochi 🤗diffusers@0.32.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Video Allegro 🤗diffusers@0.32.0, 🤗HF 1 ✔️/✖️/✔️/✔️
Video LTX 🤗diffusers@0.32.0, 🤗HF 5 ✔️/✔️/✔️/✔️
Image CogView3Plus 🤗diffusers@0.31.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image FLUX 🤗diffusers@0.30.0, 🤗HF 10+ ✔️/✔️/✔️/✔️
Image HunyuanDiT-PAG 🤗diffusers@0.30.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image AuraFlow 🤗diffusers@0.30.0, 🤗HF 1 ✔️/✖️/✔️/✔️
Image Lumina 🤗diffusers@0.30.0, 🤗HF 2 ✔️/✖️/✔️/✔️
Video CogVideoX 🤗diffusers@0.30.0, 🤗HF 4 ✔️/✔️/✔️/✔️
Audio StableAudio 🤗diffusers@0.30.0, 🤗HF 1 ✔️/✖️/✔️/✔️
Image SD3 🤗diffusers@0.29.0, 🤗HF 3 ✔️/✖️/✔️/✔️
Image HunyuanDiT 🤗diffusers@0.28.1, 🤗HF 1 ✔️/✔️/✔️/✔️
Image PixArt-Sigma 🤗diffusers@0.28.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Image Amused 🤗diffusers@0.25.0, 🤗HF 1 ✔️/✖️/✔️/✔️
Image PixArt-Alpha 🤗diffusers@0.22.0, 🤗HF 1 ✔️/✔️/✔️/✔️
Other Shap-E 🤗diffusers@0.18.0, 🤗HF 1 ✔️/✖️/✔️/✔️
Image DiT 🤗diffusers@0.12.0, 🤗HF 1 ✔️/✔️/✔️/✔️

C: Hybrid Cache (DBCache + Calibrator: TaylorSeer/DMD/FoCa/SCM); P: Parallelism (Ulysses/Ring/USP
TP/TE-P/VAE-P/2D-P/3D-P); Q: Quantization; OF: Bucket-style Layerwise Offload; @: Minimum version.

🚀Built-in Pipelines Beyond 🤗Diffusers

Modality Pipeline Series Model Sources Variants C/P/Q/OF
Video SANA-Video 2.0 (T2V/TI2V) 🎯cache-dit@1.5.5, 🤗HF 1 ✔️/✔️/✔️/✔️
Video LingBot-Video Dense 🔥 🎯cache-dit@1.5.5, 🤗HF 1 ✔️/✔️/✔️/✔️
Video MAGI-2 Preview (T2VA/I2VA) 🎯cache-dit@1.5.5, 🤗HF 2 ✖️/✔️/✖️/✔️
Image Boogu-Image (T2I/IE2I) 🔥 🎯cache-dit@1.5.5, 🤗HF 1 ✔️/✔️/✔️/✔️
Image MOSS-Image 🔥 🎯cache-dit@1.5.5, 🤗HF 2 ✔️/✔️/✔️/✔️
Image Ming-Image 0.1 🔥 🎯cache-dit@1.5.5, 🤗HF 2 ✔️/✔️/✔️/✔️
Image HiDream-O1-Image 🔥 🎯cache-dit@1.5.5, 🤗HF 1 ✔️/✔️/✔️/✔️
Video Fast-H3 🔥 🎯cache-dit@1.5.5, 🤗HF 2 ✔️/✔️/✔️/✔️
Video CausVid-Wan 🔥 🎯cache-dit@1.5.5, 🤗HF 1 ✖️/✔️/✔️/✔️
Video LongLive 2.0 (5B) 🔥 🎯cache-dit@1.5.5, 🤗HF 1 ✔️/✔️/✔️/✔️
Video LongCat-Video (A2V/AI2V) 🎯cache-dit@1.5.5, 🤗HF 1 ✔️/✔️/✔️/✔️
Video Wan2.2-S2V (S2V) 🔥 🎯cache-dit@1.5.5, 🤗HF 1 ✔️/✔️/✔️/✔️
Video MOVA 360p (T2VA/I2VA) 🔥 🎯cache-dit@1.5.5, 🤗HF 1 ✔️/✔️/✔️/✔️
Video HY-Nomas (T2V) 🔥 🎯cache-dit@1.5.5, 🤗HF 1 ✔️/✔️/✔️/✔️

Cache-DiT also provides built-in pipelines for models that are not currently included in 🤗Diffusers.

🤖Agentic Workflows

Cache-DiT provides a model-integration SKILL to help users integrate new DiT pipelines into Cache-DiT, including Cache, CP, TP, TE-P, VAE-P and carefully designed test cases. Users can use it with Coding Agents, e.g, GitHub Copilot, Claude Code, Open Code.

gh skill install vipshop/cache-dit cache-dit-model-integration --allow-hidden-dirs

Model integration: The SKILL guides the Coding Agent through: search and download the model weights (e.g. from 🤗Hugging Face or ModelScope), match the model against the corresponding 🤗Diffusers pipeline and transformer code, and then implement support for it following the SKILL's instructions.

/cache-dit-model-integration "Please download and support MiniMax-H3 in Cache-DiT."

Performance: It can also run the performance matrix for models that Cache-DiT already supports. For examples:

/cache-dit-model-integration "Run the perf matrix for MiniMax-H3 with 1/2/4/8 GPUs."

🌐Community Integration

©️Acknowledgements

Special thanks to vipshop's Computer Vision AI Team for supporting testing and deployment of this project. We learned and reused codes from: Diffusers, SGLang, vLLM-Omni, Nunchaku, xDiT and TaylorSeer.

©️Citations

@misc{cache-dit@2025,
  title={Cache-DiT: A PyTorch-native Inference Engine with Cache, Parallelism, Quantization and CPU Offload for DiTs.},
  url={https://github.com/vipshop/cache-dit.git},
  note={Open-source software available at https://github.com/vipshop/cache-dit.git},
  author={DefTruth, vipshop.com, etc.},
  year={2025}
}

Metadata

Release files for cache-dit 1.5.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for cache-dit 1.5.5
File Interpreter ABI Platform Actions
cache_dit-1.5.5-py3-none-any.whl Python 3 none any Details for cache_dit-1.5.5-py3-none-any.whl

Release files / cache_dit-1.5.5-py3-none-any.whl

Download URL cache_dit-1.5.5-py3-none-any.whl
Size 1.4 MB
Tags Python 3
SHA-256 checksum
How to use checksums
685ad24e1558747b52528c053dd291fb761c783593ef7ae9c313054c151f211c
BLAKE2b-256 checksum
How to use checksums
0978e258a764b69449d0c31ef9c92d844c5e522a3551ead7d6d60a566cbd52db
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.13

Security for cache-dit 1.5.5

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.5.5 This release

1 release file No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page