mindone 0.4.0

ONE for all, Optimal generator with No Exception.

MindSpore ONE

This repository contains SoTA algorithms, models, and interesting projects in the area of multimodal understanding and content generation.

ONE is short for "ONE for all"

News

• [2025.10.29] We release v0.4.0. This release brings 200+ new state-of-the-art AI models across language, vision, and audio domains. Test it now!
• [2025.04.10] We release v0.3.0. More than 15 SoTA generative models are added, including Flux, CogView4, OpenSora2.0, Movie Gen 30B , CogVideoX 5B~30B. Have fun!
• [2025.02.21] We support DeepSeek Janus-Pro, a SoTA multimodal understanding and generation model. See here
• [2024.11.06] v0.2.0 is released

Quick tour

To install v0.4.0, please install >=2.6.0 and run pip install mindone

Alternatively, to install the latest version from the master branch, please run.

git clone https://github.com/mindspore-lab/mindone.git
cd mindone
pip install -e .

We support state-of-the-art diffusion models for generating images, audio, and video. Let's get started using Stable Diffusion 3 as an example.

Hello MindSpore from Stable Diffusion 3!

import mindspore
from mindone.diffusers import StableDiffusion3Pipeline

pipe = StableDiffusion3Pipeline.from_pretrained(
    "stabilityai/stable-diffusion-3-medium-diffusers",
    mindspore_dtype=mindspore.float16,
)
prompt = "A cat holding a sign that says 'Hello MindSpore'"
image = pipe(prompt)[0][0]
image.save("sd3.png")

run hf diffusers on mindspore

• mindone diffusers is under active development, most tasks were tested with mindspore 2.5.0 on Ascend Atlas 800T A2 machines.
• compatibale with hf diffusers 0.32.2

component	features
pipeline	support text-to-image,text-to-video,text-to-audio tasks 160+
models	support audoencoder & transformers base models same as hf diffusers 50+
schedulers	support diffusion schedulers (e.g., ddpm and dpm solver) same as hf diffusers 35+

supported models under mindone/examples

model	inference	finetune	pretrain	organization
hunyuanvideo-i2v	✅	✖️	✖️	Tencent
wan2.1	✅	✖️	✖️	Alibaba
wan2.2	✅	✖️	✖️	Alibaba
cogview4	✅	✖️	✖️	Zhipuai
step_video_t2v	✅	✖️	✖️	StepFun
qwen2_vl	✅	✖️	✖️	Alibaba
lang_sam	✅	✖️	✖️	Luca Medeiros
sam2	✅	✖️	✖️	Meta
sparktts	✅	✖️	✖️	SparkAudio
flux	✅	✅	✖️	Black Forest Lab
janus	✅	✅	✅	DeepSeek
emu3	✅	✅	✅	BAAI
var	✅	✅	✅	ByteDance
hpcai open sora 1.2/2.0	✅	✅	✅	HPC-AI Tech
cogvideox 1.5 5B~30B	✅	✅	✅	Zhipu
open sora plan 1.3	✅	✅	✅	PKU
hunyuanvideo	✅	✅	✅	Tencent
movie gen 30B	✅	✅	✅	Meta
canny_edit	✅	✅	✅	VayneXie
mmada	✅	✅	✅	Gen-Verse
omnigen	✅	✅	✅	VectorVision
omnigen2	✅	✅	✅	VectorVision