Skip to main content

an out-of-the-box acceleration library for diffusion models

Project description


onediff is an out-of-the-box acceleration library for diffusion models, it provides:

  • Out-of-the-box acceleration for popular UIs/libs(such as HF diffusers and ComfyUI)
  • PyTorch code compilation tools and strong optimized GPU Kernels for diffusion models

For example:

News


Documentation

onediff is the abbreviation of "one line of code to accelerate diffusion models".

Use with HF diffusers and ComfyUI

Performance comparison

SDXL E2E time

  • Model stabilityai/stable-diffusion-xl-base-1.0;
  • Image size 1024*1024, batch size 1, steps 30;
  • NVIDIA A100 80G SXM4;

SVD E2E time

  • Model stabilityai/stable-video-diffusion-img2vid-xt;
  • Image size 576*1024, batch size 1, steps 25, decoder chunk size 5;
  • NVIDIA A100 80G SXM4;

Note that we haven't got a way to run SVD with TensorRT on Feb 29 2024.

Quality Evaluation

We also maintain a repository for benchmarking the quality of generation after acceleration: odeval

Community and Support

Installation

0. OS and GPU Compatibility

1. Install torch and diffusers

Note: You can choose the latest versions you want for diffusers or transformers.

python3 -m pip install "torch" "transformers==4.27.1" "diffusers[torch]==0.19.3"

2. Install a compiler backend

When considering the choice between OneFlow and Nexfort, either one is optional, and only one is needed.

  • For DiT structural models or H100 devices, it is recommended to use Nexfort.

  • For all other cases, it is recommended to use OneFlow. Note that optimizations within OneFlow will gradually transition to Nexfort in the future.

Nexfort

Install Nexfort is Optional. The detailed introduction of Nexfort is here.

python3 -m  pip install -U torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 torchao==0.1
python3 -m  pip install -U nexfort
OneFlow

Install OneFlow is Optional.

NOTE: We have updated OneFlow frequently for onediff, so please install OneFlow by the links below.

  • CUDA 11.8

    For NA/EU users

    python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu118
    

    For CN users

    python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu118
    
Click to get OneFlow packages for other CUDA versions.
  • CUDA 12.1

    For NA/EU users

    python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu121
    

    For CN users

    python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu121
    
  • CUDA 12.2

    For NA/EU users

    python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu122
    

    For CN users

    python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu122
    

3. Install onediff

  • From PyPI
python3 -m pip install --pre onediff
  • From source
git clone https://github.com/siliconflow/onediff.git
cd onediff && python3 -m pip install -e .

Or install for development:

# install for dev
cd onediff && python3 -m pip install -e '.[dev]'

# code formatting and linting
pip3 install pre-commit
pre-commit install
pre-commit run --all-files

NOTE: If you intend to utilize plugins for ComfyUI/StableDiffusion-WebUI, we highly recommend installing OneDiff from the source rather than PyPI. This is necessary as you'll need to manually copy (or create a soft link) for the relevant code into the extension folder of these UIs/Libs.

More about onediff

Architecture

Features

Functionality Details
Compiling Time About 1 minute (SDXL)
Deployment Methods Plug and Play
Dynamic Image Size Support Support with no overhead
Model Support SD1.5~2.1, SDXL, SDXL Turbo, etc.
Algorithm Support SD standard workflow, LoRA, ControlNet, SVD, InstantID, SDXL Lightning, etc.
SD Framework Support ComfyUI, Diffusers, SD-webui
Save & Load Accelerated Models Yes
Time of LoRA Switching Hundreds of milliseconds
LoRA Occupancy Tens of MB to hundreds of MB.
Device Support NVIDIA GPU 3090 RTX/4090 RTX/A100/A800/A10 etc. (Compatibility with Ascend in progress)

Acceleration for State-of-the-art models

onediff supports the acceleration for SOTA models.

  • stable: release for public usage, and has long-term support;
  • beta: release for professional usage, and has long-term support;
  • alpha: early release for expert usage, and should be careful to use;
AIGC Type Models HF diffusers ComfyUI SD web UI
Community Enterprise Community Enterprise Community Enterprise
Image SD 1.5 stable stable stable stable stable stable
SD 2.1 stable stable stable stable stable stable
SDXL stable stable stable stable stable stable
LoRA stable stable stable
ControlNet stable stable
SDXL Turbo stable stable
LCM stable stable
SDXL DeepCache alpha alpha alpha alpha
InstantID beta beta
Video SVD(stable Video Diffusion) stable stable stable stable
SVD DeepCache alpha alpha alpha alpha

Acceleration for production environment

PyTorch Module compilation

Avoid compilation time for new input shape

Avoid compilation time for online serving

Compile and save the compiled result offline, then load it online for serving

onediff Enterprise Edition

If you need Enterprise-level Support for your system or business, you can email us at contact@siliconflow.com, or contact us through the website: https://siliconflow.cn/pricing

  onediff Enterprise Edition onediff Community Edition
More Extreme and Dedicated optimization(usually another 20~100% performance gain) for the most used model Yes
Technical Support for deployment High priority support Community

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

onediff-1.2.0.dev202407220130.tar.gz (80.6 kB view details)

Uploaded Source

Built Distribution

onediff-1.2.0.dev202407220130-py3-none-any.whl (92.6 kB view details)

Uploaded Python 3

File details

Details for the file onediff-1.2.0.dev202407220130.tar.gz.

File metadata

File hashes

Hashes for onediff-1.2.0.dev202407220130.tar.gz
Algorithm Hash digest
SHA256 aa777fe5f0c7a9cf8c34d3e8c1fbdfd4fb71dbc62725325f5260e4542754cdd3
MD5 762e76a13f6e235923ba51a435a20138
BLAKE2b-256 184a0070080c226077fa8fd9a33fa46614807c3bccab82611405d90cc24f85a0

See more details on using hashes here.

File details

Details for the file onediff-1.2.0.dev202407220130-py3-none-any.whl.

File metadata

File hashes

Hashes for onediff-1.2.0.dev202407220130-py3-none-any.whl
Algorithm Hash digest
SHA256 18af695aa83527c45b884ae8b7acf786b3127e7353ebed250f6826343f46cb53
MD5 5a39eb0cc67296ef9d0b6ab38e7b4e95
BLAKE2b-256 14ed716c782df1ade4b50df93481eb59f6c13cf009cbb95de9d1307ddebf5a67

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page