This release is a pre-release and may not be stable for production use.
OneDiff
OneDiff is an out-of-the-box acceleration library for diffusion models, it provides:
- PyTorch Module compilation tools and strong optimized GPU Kernels for diffusion models
- Out-of-the-box acceleration for popular UIs/libs
For example:
News
OneDiff is the abbreviation of "one line of code to accelerate diffusion models". Here is the latest news:
- :rocket:Accelerating Stable Video Diffusion 3x faster with OneDiff DeepCache + Int8
- :rocket:Accelerating SDXL 3x faster with DeepCache and OneDiff
- :rocket:InstantID can run 1.8x Faster with OneDiff
Community and Support
Here is the introduction of OneDiff Community.
- Create an issue
- Chat in Discord:
- Email for Enterprise Edition or other business inquiries: contact@siliconflow.com
The Full Introduction of OneDiff:
About OneDiff
Architecture
OneDiff interfaces with various front-end sd frameworks upward, and uses a custom virtual machine mixed with PyTorch as the inference engine downward.
State-of-the-art performance
SDXL E2E time
- Model stabilityai/stable-diffusion-xl-base-1.0;
- Image size 1024*1024, batch size 1, steps 30;
- NVIDIA A100 80G SXM4;
SVD E2E time
- Model stabilityai/stable-video-diffusion-img2vid-xt;
- Image size 576*1024, batch size 1, steps 25, decoder chunk size 5;
- NVIDIA A100 80G SXM4;
Features
| Main Function | Details |
|---|---|
| Compiling Time | About 1 minute (SDXL) |
| Deployment Methods | Plug and Play |
| Dynamic Image Size Support | Support with no overhead |
| Model Support | SD1.5~2.1, SDXL, SDXL Turbo, etc. |
| Algorithm Support | SD standard workflow, LoRA, ControlNet, SVD, InstantID, SDXL Lightning, etc. |
| SD Framework Support | ComfyUI, Diffusers, SD-webui |
| Save & Load Accelerated Models | Yes |
| Time of LoRA Switching | Hundreds of milliseconds |
| LoRA Occupancy | Tens of MB to hundreds of MB. |
| Device Support | NVIDIA GPU 3090 RTX/4090 RTX/A100/A800/A10 etc. (Compatibility with Ascend in progress) |
Acceleration for State-of-the-art models
OneDiff supports the acceleration for SOTA models.
- stable: release for public usage, and has long-term support;
- beta: release for professional usage, and has long-term support;
- alpha: early release for expert usage, and is under active development;
| AIGC Type | Models | HF diffusers | ComfyUI | SD web UI | |||
|---|---|---|---|---|---|---|---|
| Community | Enterprise | Community | Enterprise | Community | Enterprise | ||
| Image | SD 1.5 | stable | stable | stable | stable | beta | beta |
| SD 2.1 | stable | stable | stable | stable | beta | beta | |
| SDXL | stable | stable | stable | stable | beta | beta | |
| LoRA | stable | stable | beta | ||||
| ControlNet | stable | stable | |||||
| SDXL Turbo | stable | stable | |||||
| LCM | stable | stable | |||||
| SDXL DeepCache | stable | beta | stable | beta | |||
| InstantID | stable | stable | |||||
| Video | SVD(stable Video Diffusion) | stable | beta | stable | beta | ||
| SVD DeepCache | stable | beta | stable | beta |
Note: Enterprise Edition contains all the functionality in Community Edition.
Acceleration for production environment
PyTorch Module compilation
Avoid compilation time for new input shape
Avoid compilation time for online serving
Compile and save the compiled result offline, then load it online for serving
- Save and Load the compiled graph
- Change device of the compiled graph to do multi-process serving
- Compile at one device(such as device 0), then use the compiled result to other device(such as device 1~7).
- This is for special scenes and is in the Enterprise Edition.
OneDiff Quality Evalution
We also maintain a repository for benchmarking the quality of generation after acceleration using OneDiff: OneDiffGenMetrics
OneDiff Enterprise Edition
If you need Enterprise-level Support for your system or business, you can
- subscribe to Enterprise Edition online and get all support after the order: https://siliconflow.com/onediff.html
- or send an email to contact@siliconflow.com and tell us about your user case, deployment scale, and requirements.
OneDiff Enterprise Edition can be subscripted for one month and one GPU and the cost is low.
| OneDiff Enterprise Edition | OneDiff Community Edition | |
|---|---|---|
| Multiple Resolutions | Yes(No time cost for most of the cases) | Yes(No time cost for most of the cases) |
| More Extreme and Dedicated optimization(usually another 20~100% performance gain) for the most used model | Yes | |
| Tools for specific(very large scale) server side deployment | Yes | |
| Technical Support for deployment | High priority support | Community |
| Get the experimental features | Yes |
Installation
OS and GPU support
- Linux
- If you want to use OneDiff on Windows, please use it under WSL.
- NVIDIA GPUs
OneDiff Installation
1. Install OneFlow
NOTE: We have updated OneFlow frequently for OneDiff, so please install OneFlow by the links below.
-
CUDA 11.8
For NA/EU users
python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu118
For CN users
python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu118
Click to get OneFlow packages for other CUDA versions.
-
CUDA 12.1
For NA/EU users
python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu121
For CN users
python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu121
-
CUDA 12.2
For NA/EU users
python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu122
For CN users
python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu122
2. Install torch and diffusers
Note: You can choose the latest versions you want for diffusers or transformers.
python3 -m pip install "torch" "transformers==4.27.1" "diffusers[torch]==0.19.3"
3. Install OneDiff
- From PyPI
python3 -m pip install --pre onediff
- From source
git clone https://github.com/siliconflow/onediff.git
cd onediff && python3 -m pip install -e .
NOTE: If you intend to utilize plugins for ComfyUI/StableDiffusion-WebUI, we highly recommend installing OneDiff from the source rather than PyPI. This is necessary as you'll need to manually copy (or create a soft link) for the relevant code into the extension folder of these UIs/Libs.
4. (Optional)Login huggingface-cli
python3 -m pip install huggingface_hub
~/.local/bin/huggingface-cli login
Release
-
run examples to check it works
cd onediff_diffusers_extensions python3 examples/text_to_image.py
-
bump version in these files:
.github/workflows/pub.yml src/onediff/__init__.py -
install build package
python3 -m pip install build
-
build wheel
rm -rf dist python3 -m build
-
upload to pypi
twine upload dist/*
3 maintainers
Metadata
3 maintainers
Release files for onediff 0.13.0.dev202404120125
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| onediff-0.13.0.dev202404120125.tar.gz | 73.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| onediff-0.13.0.dev202404120125-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 157.3 kB
Release files / onediff-0.13.0.dev202404120125.tar.gz
| Download URL | onediff-0.13.0.dev202404120125.tar.gz |
|---|---|
| Size | 73.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
250a62c80e94ef320f7398870c4907099f4f65875e0dc940ba31fcf598557ebf
|
|
BLAKE2b-256 checksum How to use checksums |
298c8d56af208013e775740b044c24e4f065b2caec7a0a5a02d5865e85b0a8b5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.9.19
|
Release files / onediff-0.13.0.dev202404120125-py3-none-any.whl
| Download URL | onediff-0.13.0.dev202404120125-py3-none-any.whl |
|---|---|
| Size | 84.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9d7e627f0327482c2252da16e1fbc0e382c29c3a8de4fedb6d0fa4dc1c96af96
|
|
BLAKE2b-256 checksum How to use checksums |
87a7b2d65f5a175043c3fb240e1d3627cfe0ffb41f178db4e997196ad77e519b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.9.19
|
