HCP-Diffusion V2
Old HCP-Diffusion V1 at main branch
Introduction
HCP-Diffusion is a Diffusion model toolbox built on top of the 🐱 RainbowNeko Engine.
It features a clean code structure and a flexible Python-based configuration file, making it easier to conduct and manage complex experiments. It includes a wide variety of training components, and compared to existing frameworks, it's more extensible, flexible, and user-friendly.
HCP-Diffusion allows you to use a single .py config file to unify training workflows across popular methods and model architectures, including Prompt-tuning (Textual Inversion), DreamArtist, Fine-tuning, DreamBooth, LoRA, ControlNet, ....
Different techniques can also be freely combined.
This framework also implements DreamArtist++, an upgraded version of DreamArtist based on LoRA. It enables high generalization and controllability with just a single image for training.
Compared to the original DreamArtist, it offers better stability, image quality, controllability, and faster training.
Installation
Install pytorch
Install via pip:
pip install hcpdiff
# Initialize configuration
hcpinit
Install from source:
git clone https://github.com/7eu7d7/HCP-Diffusion.git
cd HCP-Diffusion
pip install -e .
# Initialize configuration
hcpinit
Use xFormers to reduce memory usage and accelerate training:
# Choose the appropriate xformers version for your PyTorch version
pip install xformers==?
🚀 Python Configuration Files
RainbowNeko Engine supports configuration files written in a Python-like syntax. This allows users to call functions and classes directly within the configuration file, with function parameters inheritable from parent configuration files. The framework automatically handles the formatting of these configuration files.
For example, consider the following configuration file:
dict(
layer=Linear(in_features=4, out_features=4)
)
During parsing, this will be automatically compiled into:
dict(
layer=dict(_target_=Linear, in_features=4, out_features=4)
)
After parsing, the framework will instantiate the components accordingly. This means users can write configuration files using familiar Python syntax.
✨ Features
Features
📦 Model Support
| Model Name | Status |
|---|---|
| Stable Diffusion 1.5 | ✅ Supported |
| Stable Diffusion XL (SDXL) | ✅ Supported |
| PixArt | ✅ Supported |
| FLUX | ✅ Supported |
| Stable Diffusion 3 (SD3) | 🚧 In Development |
🧠 Fine-Tuning Capabilities
| Feature | Description/Support |
|---|---|
| LoRA Layer-wise Configuration | ✅ Supported (including Conv2d) |
| Layer-wise Fine-Tuning | ✅ Supported |
| Multi-token Prompt-Tuning | ✅ Supported |
| Layer-wise Model Merging | ✅ Supported |
| Custom Optimizers | ✅ Supported (Lion, DAdaptation, pytorch-optimizer, etc.) |
| Custom LR Schedulers | ✅ Supported |
🧩 Extension Method Support
| Method | Status |
|---|---|
| ControlNet (including training) | ✅ Supported |
| DreamArtist / DreamArtist++ | ✅ Supported |
| Token Attention Adjustment | ✅ Supported |
| Max Sentence Length Extension | ✅ Supported |
| Textual Inversion (Custom Tokens) | ✅ Supported |
| CLIP Skip | ✅ Supported |
🚀 Training Acceleration
| Tool/Library | Supported Modules |
|---|---|
| 🤗 Accelerate | ✅ Supported |
| Colossal-AI | ✅ Supported |
| xFormers | ✅ Supported (UNet and text encoder) |
🗂 Dataset Support
| Feature | Description |
|---|---|
| Aspect Ratio Bucket (ARB) | ✅ Auto-clustering supported |
| Multi-source / Multi-dataset | ✅ Supported |
| LMDB | ✅ Supported |
| webdataset | ✅ Supported |
| Local Attention Enhancement | ✅ Supported |
| Tag Shuffling & Dropout | ✅ Multiple tag editing strategies |
📉 Supported Loss Functions
| Loss Type | Description |
|---|---|
| Min-SNR | ✅ Supported |
| SSIM | ✅ Supported |
| GWLoss | ✅ Supported |
🌫 Supported Diffusion Strategies
| Strategy Type | Status |
|---|---|
| DDPM | ✅ Supported |
| EDM | ✅ Supported |
| Flow Matching | ✅ Supported |
🧠 Automatic Evaluation (Step Selection Assistant)
| Feature | Description/Status |
|---|---|
| Image Preview | ✅ Supported (workflow preview) |
| FID | 🚧 In Development |
| CLIP Score | 🚧 In Development |
| CCIP Score | 🚧 In Development |
| Corrupt Score | 🚧 In Development |
⚡️ Image Generation
| 功能 | 描述/支持情况 |
|---|---|
| Batch Generation | ✅ Supported |
| Generate from Prompt Dataset | ✅ Supported |
| Image to Image | ✅ Supported |
| Inpaint | ✅ Supported |
| Token Weight | ✅ Supported |
Getting Started
Training
HCP-Diffusion provides training scripts based on 🤗 Accelerate.
# Multi-GPU training, configure GPUs in cfgs/launcher/multi.yaml
hcp_train --cfg cfgs/train/py/your_config.py
# Single-GPU training, configure GPU in cfgs/launcher/single.yaml
hcp_train_1gpu --cfg cfgs/train/py/your_config.py
You can also override config items via command line:
# Override base model path
hcp_train --cfg cfgs/train/py/your_config.py model.wrapper.models.ckpt_path=pretrained_model_path
Image Generation
Use the workflow defined in the Python config to generate images:
hcp_run --cfg cfgs/workflow/text2img.py
Or override parameters via command line:
hcp_run --cfg cfgs/workflow/text2img_cli.py \
pretrained_model=pretrained_model_path \
prompt='positive_prompt' \
negative_prompt='negative_prompt' \
seed=42
📚 Tutorials
- 🧠 Model Training Guide
- 🔧 LoRA Training Tutorial
- 🎨 Image Generation Guide
- ⚙️ Configuration File Explanation
- 🧩 Model Format Explanation
Contributing
We welcome contributions to support more models and features.
Team
Maintained by HCP-Lab at Sun Yat-sen University.
Citation
@article{DBLP:journals/corr/abs-2211-11337,
author = {Ziyi Dong and
Pengxu Wei and
Liang Lin},
title = {DreamArtist: Towards Controllable One-Shot Text-to-Image Generation
via Positive-Negative Prompt-Tuning},
journal = {CoRR},
volume = {abs/2211.11337},
year = {2022},
doi = {10.48550/arXiv.2211.11337},
eprinttype = {arXiv},
eprint = {2211.11337},
}
1 maintainer
Metadata
1 maintainer
Release files for hcpdiff 2.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| hcpdiff-2.6.tar.gz | 92.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| hcpdiff-2.6-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 225.1 kB
Release files / hcpdiff-2.6.tar.gz
| Download URL | hcpdiff-2.6.tar.gz |
|---|---|
| Size | 92.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
91990e098fe998458cc70dd735fa77ba318a7e856e35dfe74b8dc43de43c3b54
|
|
BLAKE2b-256 checksum How to use checksums |
68bad5043b4881423a72ee3377fec2e728668f89b8559867bf51459adde76a14
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.25
|
Release files / hcpdiff-2.6-py3-none-any.whl
| Download URL | hcpdiff-2.6-py3-none-any.whl |
|---|---|
| Size | 132.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
e152bf2bf172ecffee54eaf1b8997cfcb5e898ce5eb2cda103a4904d50e93eef
|
|
BLAKE2b-256 checksum How to use checksums |
ffabb98d02dd0b0cf365cbfcf8d4ae0a83ca4443dedb069e1d0853d6a5af7309
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.25
|
