Skip to main content

an out-of-the-box acceleration library for diffusion models

Project description


Docker image build CI testing

OneDiff

OneDiff is an out-of-the-box acceleration library for diffusion models, it provides:

News

OneDiff is the abbreviation of "one line of code to accelerate diffusion models". Here is the latest news:

Community and Support

Here is the introduction of OneDiff Community.


The Full Introduction of OneDiff:

About OneDiff

Architecture

OneDiff interfaces with various front-end sd frameworks upward, and uses a custom virtual machine mixed with PyTorch as the inference engine downward.

State-of-the-art performance

SDXL E2E time

  • Model stabilityai/stable-diffusion-xl-base-1.0;
  • Image size 1024*1024, batch size 1, steps 30;
  • NVIDIA A100 80G SXM4;

SVD E2E time

  • Model stabilityai/stable-video-diffusion-img2vid-xt;
  • Image size 576*1024, batch size 1, steps 25, decoder chunk size 5;
  • NVIDIA A100 80G SXM4;

Features

Main Function Details
Compiling Time About 1 minute (SDXL)
Deployment Methods Plug and Play
Dynamic Image Size Support Support with no overhead
Model Support SD1.5~2.1, SDXL, SDXL Turbo, etc.
Algorithm Support SD standard workflow, LoRA, ControlNet, SVD, InstantID, SDXL Lightning, etc.
SD Framework Support ComfyUI, Diffusers, SD-webui
Save & Load Accelerated Models Yes
Time of LoRA Switching Hundreds of milliseconds
LoRA Occupancy Tens of MB to hundreds of MB.
Device Support NVIDIA GPU 3090 RTX/4090 RTX/A100/A800/A10 etc. (Compatibility with Ascend in progress)

Acceleration for State-of-the-art models

OneDiff supports the acceleration for SOTA models.

  • stable: release for public usage, and has long-term support;
  • beta: release for professional usage, and has long-term support;
  • alpha: early release for expert usage, and is under active development;
AIGC Type Models HF diffusers ComfyUI SD web UI
Community Enterprise Community Enterprise Community Enterprise
Image SD 1.5 stable stable stable stable beta beta
SD 2.1 stable stable stable stable beta beta
SDXL stable stable stable stable beta beta
LoRA stable stable beta
ControlNet stable stable
SDXL Turbo stable stable
LCM stable stable
SDXL DeepCache stable beta stable beta
InstantID stable stable
Video SVD(stable Video Diffusion) stable beta stable beta
SVD DeepCache stable beta stable beta

Note: Enterprise Edition contains all the functionality in Community Edition.

Acceleration for production environment

PyTorch Module compilation

Avoid compilation time for new input shape

Avoid compilation time for online serving

Compile and save the compiled result offline, then load it online for serving

OneDiff Quality Evalution

We also maintain a repository for benchmarking the quality of generation after acceleration using OneDiff: OneDiffGenMetrics

OneDiff Enterprise Edition

If you need Enterprise-level Support for your system or business, you can

OneDiff Enterprise Edition can be subscripted for one month and one GPU and the cost is low.

  OneDiff Enterprise Edition OneDiff Community Edition
Multiple Resolutions Yes(No time cost for most of the cases) Yes(No time cost for most of the cases)
More Extreme and Dedicated optimization(usually another 20~100% performance gain) for the most used model Yes
Tools for specific(very large scale) server side deployment Yes
Technical Support for deployment High priority support Community
Get the experimental features Yes

Installation

OS and GPU support

  • Linux
    • If you want to use OneDiff on Windows, please use it under WSL.
  • NVIDIA GPUs

OneDiff Installation

1. Install OneFlow

NOTE: We have updated OneFlow frequently for OneDiff, so please install OneFlow by the links below.

  • CUDA 11.8

    For NA/EU users

    python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu118
    

    For CN users

    python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu118
    
Click to get OneFlow packages for other CUDA versions.
  • CUDA 12.1

    For NA/EU users

    python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu121
    

    For CN users

    python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu121
    
  • CUDA 12.2

    For NA/EU users

    python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu122
    

    For CN users

    python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu122
    

2. Install torch and diffusers

Note: You can choose the latest versions you want for diffusers or transformers.

python3 -m pip install "torch" "transformers==4.27.1" "diffusers[torch]==0.19.3"

3. Install OneDiff

  • From PyPI
python3 -m pip install --pre onediff
  • From source
git clone https://github.com/siliconflow/onediff.git
cd onediff && python3 -m pip install -e .

NOTE: If you intend to utilize plugins for ComfyUI/StableDiffusion-WebUI, we highly recommend installing OneDiff from the source rather than PyPI. This is necessary as you'll need to manually copy (or create a soft link) for the relevant code into the extension folder of these UIs/Libs.

4. (Optional)Login huggingface-cli

python3 -m pip install huggingface_hub
 ~/.local/bin/huggingface-cli login

Release

  • run examples to check it works

    cd onediff_diffusers_extensions
    python3 examples/text_to_image.py
    
  • bump version in these files:

    .github/workflows/pub.yml
    src/onediff/__init__.py
    
  • install build package

    python3 -m pip install build
    
  • build wheel

    rm -rf dist
    python3 -m build
    
  • upload to pypi

    twine upload dist/*
    

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

onediff-0.13.0.dev202404080125.tar.gz (70.7 kB view details)

Uploaded Source

Built Distribution

onediff-0.13.0.dev202404080125-py3-none-any.whl (82.2 kB view details)

Uploaded Python 3

File details

Details for the file onediff-0.13.0.dev202404080125.tar.gz.

File metadata

File hashes

Hashes for onediff-0.13.0.dev202404080125.tar.gz
Algorithm Hash digest
SHA256 423fa47c48bc2b3a93c267bfb4f8cb70b2629c0a55d691d3eff9281f763db1dd
MD5 2a493d71a4c9c25affc180da6034329b
BLAKE2b-256 bbd5a529580071aefe2b7896ea3947f913a8c5de58bd11885c66e4d7adbac04d

See more details on using hashes here.

File details

Details for the file onediff-0.13.0.dev202404080125-py3-none-any.whl.

File metadata

File hashes

Hashes for onediff-0.13.0.dev202404080125-py3-none-any.whl
Algorithm Hash digest
SHA256 138d28df6c258654be5ee94cdfb4e0e9ce1d354157e1e21f5fe3cb796ee49f2b
MD5 181d65e236d399612ccec91484134bdf
BLAKE2b-256 5483d537635041e8f9cdc1698f9a973c7eb58cedaa82a6bfb4c84d52b3e58da2

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page