Project Status: Active Development

Thank you for the incredible support and over 30,000 downloads!

ollamadiffuser is in active development. v2.3.0: 106 models — 44 native on Apple Silicon through MLX (38 image models, 6 LTX-2 video packs) and 62 through diffusers — with one-command image, edit and video jobs for scripts and agents, Qwen-Image-2.1 editing and its official 8-step Turbo on a Mac, video on Mac and NVIDIA, LoRAs on both backends, and a compile-free install with a one-line curl | sh installer. See What's New below. Part of the LocalKinAI ecosystem.

🆕 What's New

v2.3.0 — one command for agents

Upgrade with pip install -U ollamadiffuser; full notes in CHANGELOG.md. A failed generation is now an error from the API rather than a 200 with a picture of the error, and on a Mac a model that won't fit in free memory isn't loaded unless forced.

ollamadiffuser generate / edit / video make a picture, an edit or a clip in one command, and print the file (--json for scripts and agents). The job goes to a server that already holds the model, or loads it for the job. See Using OllamaDiffuser from agents.
ps --json, list --json, stop --port N / --all; MCP gains edit_image, generate_video and list_servers; a Claude Code skill in integrations/claude-code/.
On a Mac, a model that won't fit in free memory isn't loaded (--force to insist), and a failed generation is now an error rather than a picture of one.

v2.2.0 — Qwen-Image-2.1 edits and its Turbo on a Mac

The first PyPI release since 2.1.0. Upgrade with pip install -U ollamadiffuser, then ollamadiffuser enable mlx on a Mac (mflux 0.22); full notes in CHANGELOG.md.

Edit with Qwen-Image-2.1 on Apple Silicon. qwen-image-2.1-edit-mlx takes up to ten reference pictures, a mask, an edit strength and prompt rewriting, from the same weights as qwen-image-2.1-mlx. Its ControlNet (qwen-image-2.1-controlnet-mlx: canny, depth, pose and five more, plus inpainting) and Qwen-Image-Edit-2511 (qwen-image-edit-2511-mlx) run natively too. Needs mflux 0.22, now the minimum.
Qwen-Image-2.1 in 6 steps. Load Viggle's turbo LoRA and pass scheduler=viggle_turbo — see LoRAs on Apple Silicon.
New: ming-image-design-mlx (UI, posters and infographics with transparent backgrounds, MIT); Qwen's own 8-step Qwen-Image-2.1-Turbo on both platforms — qwen-image-2.1-turbo-mlx (sampled on the schedule the checkpoint ships with, which mflux has no entry for; 32 s an image at 1024² on an M3 Ultra) and qwen-image-2.1-turbo through diffusers 0.41 — and qwen-image-2.1 on NVIDIA.
Fixed: qwen-image-edit-mlx redrew the picture instead of editing it — it was built as the text-to-image class. The MCP server didn't start with mcp 2.x, which ollamadiffuser enable mcp installs. MLX models hold MLX's buffer cache to 2 GB (OLLAMADIFFUSER_MLX_CACHE_LIMIT_GB): by default MLX lets it grow to the size of the machine, and one 1024² Qwen-Image-2.1-Turbo image took a 96 GB Mac from 28 GB free to under 5 GB, though it needed 15.5 GB.
Better: pull links files another installed entry already downloaded from the same repo, so an editor next to its generator costs no second download. ollamadiffuser ps lists every running server, not just the last one started. MCP's generate_image also returns the saved file's path as text, and uses a running server that already holds the model instead of loading a second copy.

v2.1.1 — ComfyUI nodes

Three nodes that let a ComfyUI workflow use OllamaDiffuser's models — Qwen-Image-2.1, FLUX.2 klein, Kontext, Boogu, LTX-2 on MLX — as ordinary IMAGE / VIDEO steps, through the HTTP API, with no second copy of the weights. See ComfyUI Nodes.

v2.1.0 — video and LoRAs on every platform, 40 more models

The first PyPI release since 2.0.17, so it also brings 2.0.18–2.0.26 below to pip. Full notes in CHANGELOG.md.

🎬 Video on NVIDIA. A new diffusers-video path runs wan2.1-t2v-1.3b, wan2.2-ti2v-5b and ltx-2.3-distilled (with its soundtrack) behind the same POST /api/generate/video as LTX-2 on Apple Silicon. On the Mac, LTX-2 can now follow a reference video — a pose skeleton, depth or edges — for motion no prompt can describe, and runs offline.
🎛️ LoRAs on Apple Silicon. ollamadiffuser lora load works on -mlx models: the model is rebuilt with the LoRA baked in, and LoRAs stack — a speed LoRA and a style LoRA together. FLUX.1, FLUX.2 klein, Z-Image, Qwen-Image, Krea 2, ERNIE. LTX video takes style and motion LoRAs too.
🖼️ New models. qwen-image-2.1-mlx (108 s per 1024² image on an M3 Ultra, faster than ComfyUI; research licence), qwen-image-edit-2511, the undistilled bases z-image, flux.2-klein-base-4b, ernie-image, krea-2-raw, flux.1-krea-dev, the anime fine-tune z-anime, seedvr2-7b-mlx, z-image-turbo-controlnet-mlx (one ControlNet for canny, depth, pose, HED, MLSD), realvisxl-v5.
🖥️ Not just for Macs. Nine models that had been Mac-only now have diffusers entries for NVIDIA: z-image, flux.2-klein-base-4b, flux.2-klein-9b, ernie-image, ernie-image-turbo, krea-2-turbo, krea-2-raw, qwen-image, flux.1-krea-dev.
♻️ Smaller downloads. pull hard-links weights already in the Hugging Face cache (from ComfyUI, mflux or hf download) instead of downloading them again, and entries stop fetching files their loader never opens — krea-2-turbo-mlx 62 → 36 GB, ernie-image-turbo-mlx 32 → 24 GB, Z-Anime 17.5 GB of a 202 GB repo.
🔌 New endpoints. POST /api/face/compare (is it still the same person?), several reference pictures on /api/generate/img2img, control_type on /api/generate/controlnet.
Upgrading: pip install -U ollamadiffuser. Two new dependencies, av and ftfy, are prebuilt wheels. The MLX backend needs mflux 0.20.0 (ollamadiffuser enable mlx). Licence correction: flux.2-klein-9b-mlx is FLUX Non-Commercial, not Apache 2.0.

v2.0.26 — the registry pulls what the loader loads

qwen-image-mlx / qwen-image-edit-mlx were pulling 58 GB of a checkpoint mflux never loads (now Qwen-Image-2512 / Qwen-Image-Edit-2509), and seedvr2-3b-mlx fetched a 3.4 GB fp8 file it never opened. ollamadiffuser enable gguf now builds with CUDA on NVIDIA machines instead of silently producing a CPU build. New: CONTRIBUTING.md — adding a model is a data-only PR.

v2.0.22 — video, natively: LTX-2 on Apple Silicon

Six new entries (ltx-2.3-mlx-{q4,q8,bf16}, ltx-2.5-mlx-{q4,q8,bf16}) drive ltx-2-mlx for text-to-video with audio, image-to-video and audio-to-video — plus a POST /api/generate/video endpoint and engine.generate_video(). See the video section.

v2.0.21 — eight more MLX families: Krea 2, Boogu, ERNIE-Image, Lens, Ideogram 4, FIBO, FIBO-Edit, SeedVR2

mflux grew a lot of model families after our May line, and the registry had not caught up. Eight new entries — 22 MLX entries total — covering photographic turbo models (Boogu 4-step with bilingual EN/ZH text, ERNIE-Image, Krea 2), a 4-step Microsoft model with a 20B text encoder (Lens), typography (Ideogram 4), JSON-prompted generation and editing (FIBO, FIBO-Edit) and the best open upscaler (SeedVR2). Each is a class and an alias: mflux resolves the config from the alias itself, so adding a family is data plus one line of routing.

v2.0.20 — One-line `curl | sh` installer (no system Python needed)

curl -fsSL .../install.sh | sh now installs a fully isolated OllamaDiffuser via uv: a standalone Python + the compile-free core under ~/.ollamadiffuser, with a launcher on your PATH. It doesn't touch your system Python, needs no compiler, and auto-enables MLX on Apple Silicon — the Ollama-style zero-friction install.

v2.0.19 — Model registry is now data (`models.yaml`)

The 59 built-in models moved from a 1500-line Python dict into a bundled models.yaml. Adding a model is now a pure-data PR — no code. Migration is zero-loss (snapshot-pinned in tests).

v2.0.18 — Compile-free default install + `enable` command

The default pip install ollamadiffuser is now compile-free (prebuilt wheels only — no CMake, no CUDA toolchain). Optional backends are opt-in by name via a new command: ollamadiffuser enable mlx | gguf | mcp — no shell-quoted [extras] to get wrong, and enable gguf sets the right CMAKE_ARGS (Metal on Mac) for you. The curl | sh installer now defaults to the lean core and auto-enables MLX on Apple Silicon.

v2.0.17 — MLX Phase 2.5: FLUX.1 family completion

MLXStrategy now covers the full FLUX.1 family on Apple Silicon: flux1-fill (inpaint/outpaint), flux1-redux (image variation), flux1-depth (depth-conditioned), flux1-controlnet (canny + upscaler). Five new registry entries — 14 MLX entries total (see the MLX section below).

v2.0.15–v2.0.16 — MLX Backend (Phases 1 + 2)

New MLXStrategy routes FLUX.1 / FLUX.2 / Z-Image / Qwen-Image / Kontext through mflux for native Apple Silicon inference. Typically 2-3× faster than the PyTorch + MPS path. Install with pip install 'ollamadiffuser[mlx]'. Tracks #7.

v2.0.14 — Diffusers Pipeline Additions

flux.1-kontext-dev (PyTorch) — 12B instruction-based image editing. Pass an input image + edit prompt; the model rewrites the image.
chroma1-hd — 8.9B Apache-2.0 base T2I (FLUX-schnell derivative). Rare commercial-friendly license at this quality tier.

v2.0.13 — Bug Fixes + Discussions

Fixed ollamadiffuser recommend crash on CUDA hosts (PyTorch attribute typo).
GitHub Discussions enabled: https://github.com/LocalKinAI/ollamadiffuser/discussions
18 broken org-name URLs fixed across PyPI metadata, README, and guides.

See CHANGELOG.md for the full history (back to v1.0.0, May 2025).

OllamaDiffuser 🎨

Local AI Image Generation with OllamaDiffuser

OllamaDiffuser simplifies local deployment of Stable Diffusion, FLUX, CogView4, Kolors, SANA, PixArt-Sigma, and 40+ other AI image generation models. An intuitive local SD tool inspired by Ollama's simplicity - perfect for local diffuser workflows with CLI, web UI, and LoRA support.

🌐 Website: ollamadiffuser.com | 📦 PyPI: pypi.org/project/ollamadiffuser

Upgrading from v1.x? v2.0 is a major rewrite requiring Python 3.10+. Run pip install --upgrade ollamadiffuser (then ollamadiffuser enable gguf/mlx if you need those backends) and see the Migration Guide below.

🚀 Quick Start

Zero-dependency install (recommended — no Python setup needed)

curl -fsSL https://raw.githubusercontent.com/LocalKinAI/ollamadiffuser/main/install.sh | sh

This installs an isolated Python + OllamaDiffuser under ~/.ollamadiffuser (via uv — a single static binary) and drops an ollamadiffuser launcher on your PATH. It never touches your system Python, needs no pip/venv/compiler, and on Apple Silicon auto-enables the MLX backend. Uninstall is rm -rf ~/.ollamadiffuser.

Already have a Python env?

The default install is compile-free — prebuilt wheels only, no CMake and no CUDA toolchain. Optional backends are opt-in by name, so you never fight bracket-quoting in your shell.

pip install ollamadiffuser
ollamadiffuser recommend   # Find which models fit your hardware

Mac / Apple Silicon (recommended — typically 2-3× faster):

pip install ollamadiffuser
ollamadiffuser enable mlx  # native MLX backend, still compile-free

OpenClaw / Agent users:

pip install ollamadiffuser
ollamadiffuser enable mcp  # Model Context Protocol server deps
ollamadiffuser mcp         # start the server

Low-VRAM / GGUF (advanced — compiles a native extension):

pip install ollamadiffuser
ollamadiffuser enable gguf                 # sets Metal/CUDA build flags for you
ollamadiffuser pull flux.1-dev-gguf-q4ks   # only 6GB VRAM needed
ollamadiffuser run  flux.1-dev-gguf-q4ks

Why enable instead of pip install "ollamadiffuser[gguf]"? The GGUF backend compiles a native library (stable-diffusion-cpp-python). ollamadiffuser enable gguf runs that build with the right CMAKE_ARGS for your platform (Metal on Mac) so you never have to remember them — and there are no shell-quoted [extras] to get wrong. The bracketed extras ([gguf], [mlx], [mcp], [full]) still work if you prefer them.

Most models work without any token -- just install and go. See Hugging Face Authentication when you want gated models like FLUX.1-dev or SD 3.5.

✨ Features

🏗️ Strategy Architecture: Clean per-model strategy pattern (SD1.5, SDXL, FLUX, SD3, ControlNet, Video, LTX-2 video (MLX), HiDream, GGUF, MLX, Generic)
🌐 60+ Models: FLUX.1/2, SD 3.5, SDXL Lightning, CogView4, Kolors, SANA, PixArt-Sigma, Z-Image, Qwen-Image, Chroma1, and more
🔌 Generic Pipeline: Add new diffusers models via registry config alone -- no code changes needed. Built-in models live in models.yaml (data, not code) — contributing a model is a pure-data PR against that file.
🖼️ img2img & Inpainting: Image-to-image and inpainting support across SD1.5, SDXL, and the API/Web UI
⚡ Async API: Non-blocking FastAPI server using asyncio.to_thread for GPU operations
🎲 Random Seeds: Reproducible generation with explicit seeds, random by default
🎛️ ControlNet Support: Precise image generation control with 10+ control types (PyTorch + MLX)
🔄 LoRA Integration: Dynamic LoRA loading and management
🔌 MCP & OpenClaw: Model Context Protocol server for AI assistant integration (OpenClaw, Claude Code, Cursor)
🧩 ComfyUI nodes: draw, edit and film with OllamaDiffuser's models from inside a ComfyUI graph (integrations/comfyui/)
🍎 Apple Silicon, two paths:
- MLX backend via mflux — 38 native MLX entries (FLUX.1 family, FLUX.1 Krea, FLUX.2 Klein, Z-Image, Qwen-Image, Qwen-Image-2.1 with editing and a ControlNet, Kontext, Fill, Redux, Depth, ControlNet, and since v2.0.21 Krea 2, Boogu, ERNIE-Image, Lens, Ideogram 4, FIBO, FIBO-Edit, SeedVR2, Ming-Image). Typically 2-3× faster than the PyTorch + MPS path on M-series.
- PyTorch + MPS — full diffusers pipeline support with per-model dtype handling (float16/bfloat16, NaN sanitization), GGUF Metal acceleration, and ollamadiffuser recommend for hardware-aware model suggestions.
📦 Smart Downloads: ollamadiffuser pull downloads only diffusers pipeline files — skips root-level checkpoints, ONNX/Flax exports, and safety_checker. Saves 10–200 GB per model.
♻️ Reuses what's already downloaded: if a model is already in the Hugging Face cache (from ComfyUI, mflux, hf download…), ollamadiffuser pull hard-links it instead of downloading — seconds instead of tens of GB, and no second copy on disk.
📦 GGUF Support: Memory-efficient quantized models (3GB VRAM minimum!) with CUDA and Metal acceleration
🌐 Multiple Interfaces: CLI, Python API, Web UI, and REST API
📦 Model Management: Easy installation and switching between models
⚡ Performance Optimized: Memory-efficient with GPU acceleration
🧪 Test Suite: 243 tests across settings, registry, engine, API, MPS, MLX, LTX-2 video, and MCP

Option 1: Install from PyPI (Recommended)

# Install from PyPI
pip install ollamadiffuser

# Pull and run a model
ollamadiffuser pull flux.1-schnell
ollamadiffuser run flux.1-schnell

# Generate via API (seed is optional for reproducibility)
curl -X POST http://localhost:8000/api/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "A beautiful sunset", "seed": 12345}' \
  --output image.png

🔄 Update to Latest Version

Always use the latest version for the newest features and bug fixes:

# Update to latest version
pip uninstall ollamadiffuser
pip install --no-cache-dir ollamadiffuser

This ensures you get:

🐛 Latest bug fixes
✨ New features and improvements
🚀 Performance optimizations
🔒 Security updates

GGUF Quick Start (Low VRAM)

# Enable the GGUF backend (compiles once; sets the right build flags for you)
ollamadiffuser enable gguf

# Download a memory-efficient GGUF model (3GB+ VRAM)
ollamadiffuser pull flux.1-dev-gguf-q4ks

# Generate with reduced memory usage
ollamadiffuser run flux.1-dev-gguf-q4ks

Apple Silicon Quick Start (Mac Mini / MacBook)

# See which models fit your Mac
ollamadiffuser recommend

# Fast single-step model (<6GB)
ollamadiffuser pull sdxl-turbo
ollamadiffuser run sdxl-turbo

# Native MLX path — typically 2-3x faster than PyTorch+MPS (auto-enabled
# by the curl|sh installer; run this if you installed via pip)
ollamadiffuser enable mlx
ollamadiffuser pull flux.1-schnell-mlx
ollamadiffuser run flux.1-schnell-mlx

# GGUF with Metal acceleration (6GB, great quality)
ollamadiffuser enable gguf   # sets CMAKE_ARGS=-DSD_METAL=ON under the hood
ollamadiffuser pull flux.1-dev-gguf-q4ks
ollamadiffuser run flux.1-dev-gguf-q4ks

Option 2: Development Installation

# Clone the repository
git clone https://github.com/LocalKinAI/ollamadiffuser.git
cd ollamadiffuser

# Install dependencies
pip install -e .

Basic Usage

# Check version
ollamadiffuser -V

# Install a model
ollamadiffuser pull stable-diffusion-1.5

# Run the model (loads and starts API server)
ollamadiffuser run stable-diffusion-1.5

# Generate an image via API
curl -X POST http://localhost:8000/api/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "a beautiful sunset over mountains"}' \
  --output image.png

# Start web interface
ollamadiffuser --mode ui

open http://localhost:8001

ControlNet Quick Start

# Install ControlNet model
ollamadiffuser pull controlnet-canny-sd15

# Run ControlNet model (loads and starts API server)
ollamadiffuser run controlnet-canny-sd15

# Generate with control image
curl -X POST http://localhost:8000/api/generate/controlnet \
  -F "prompt=a beautiful landscape" \
  -F "control_image=@your_image.jpg"

🔑 Hugging Face Authentication

Do you need a Hugging Face token? It depends on which models you want to use!

Models that DON'T require a token -- ready to use right away:

FLUX.1-schnell, Stable Diffusion 1.5, DreamShaper, PixArt-Sigma, SANA 1.5, most ControlNet models

Models that DO require a token:

FLUX.1-dev, Stable Diffusion 3.5, some premium LoRAs

Setup (only needed for gated models):

# 1. Create account at https://huggingface.co and generate an access token
# 2. Accept license on the model page (e.g. FLUX.1-dev, SD 3.5)
# 3. Set your token
export HF_TOKEN=your_token_here

# 4. Now you can access gated models
ollamadiffuser pull flux.1-dev
ollamadiffuser pull stable-diffusion-3.5-medium

Tips: Use "read" permissions for the token. Your token stays local -- never shared with OllamaDiffuser servers. Add export HF_TOKEN=... to ~/.bashrc or ~/.zshrc to make it permanent.

🎯 Supported Models

Choose from 40+ models spanning every major architecture:

Core Models

Model	Type	Steps	VRAM	Commercial	License
`flux.1-schnell`	flux	4	16GB+	✅	Apache 2.0
`flux.1-dev`	flux	20	20GB+	❌	Non-commercial
`stable-diffusion-3.5-medium`	sd3	28	8GB+	⚠️	Stability AI
`stable-diffusion-3.5-large`	sd3	28	12GB+	⚠️	Stability AI
`stable-diffusion-3.5-large-turbo`	sd3	4	12GB+	⚠️	Stability AI
`stable-diffusion-xl-base`	sdxl	50	6GB+	⚠️	CreativeML
`stable-diffusion-1.5`	sd15	50	4GB+	⚠️	CreativeML

Next-Generation Models

Model	Origin	Params	Steps	VRAM	Commercial	License
`flux.2-dev`	Black Forest Labs	32B	28	14GB+	❌	Non-commercial
`flux.2-klein-4b`	Black Forest Labs	4B	28	10GB+	✅	Apache 2.0
`z-image-turbo`	Alibaba (Tongyi)	6B	8	10GB+	✅	Apache 2.0
`z-image`	Alibaba (Tongyi)	6B	50	16GB+	✅	Apache 2.0
`z-anime`	SeeSee21 (Z-Image fine-tune)	6B	40	16GB+	✅	Apache 2.0
`flux.2-klein-base-4b`	Black Forest Labs	4B	50	10GB+	✅	Apache 2.0
`flux.2-klein-9b`	Black Forest Labs	9B	4	20GB+	❌	Non-commercial
`flux.1-krea-dev`	Black Forest Labs × Krea	12B	28	20GB+	❌	Non-commercial
`krea-2-turbo` / `krea-2-raw`	Krea	12B	8 / 28	24GB+	❌	Krea 2 Community
`ernie-image-turbo` / `ernie-image`	Baidu	8B	8 / 50	20GB+	✅	Apache 2.0
`qwen-image`	Alibaba (Qwen), 2512	20B	50	24GB+ (offload)	✅	Apache 2.0
`qwen-image-2.1` / `qwen-image-2.1-turbo`	Alibaba (Qwen), diffusers 0.41+	7B + 8B encoder	40 / 8	24GB+ (offload)	❌	Qwen Research
`sana-1.5`	NVIDIA	1.6B	20	8GB+	✅	Apache 2.0
`cogview4`	Zhipu AI	6B	50	12GB+	✅	Apache 2.0
`kolors`	Kuaishou	8.6B	50	8GB+	✅	Kolors License
`hunyuan-dit`	Tencent	1.5B	50	6GB+	✅	Tencent Community
`lumina-2`	Alpha-VLLM	2B	30	8GB+	✅	Apache 2.0
`pixart-sigma`	PixArt	0.6B	20	6GB+	✅	Open
`auraflow`	Fal	6.8B	50	12GB+	✅	Apache 2.0
`omnigen`	BAAI	3.8B	50	12GB+	✅	MIT

Fast / Turbo Models

Model	Steps	VRAM	Notes
`sdxl-turbo`	1	6GB+	Single-step distilled SDXL
`sdxl-lightning-4step`	4	6GB+	ByteDance, single-file checkpoint, custom scheduler
`stable-diffusion-3.5-large-turbo`	4	12GB+	Distilled SD 3.5 Large
`z-image-turbo`	8	10GB+	Alibaba 6B turbo

Community Fine-Tunes

Model	Base	Notes
`realvisxl-v5`	SDXL	Photorealistic, current release (pulls only the 7 GB fp16 set)
`realvisxl-v4`	SDXL	Photorealistic, very popular
`dreamshaper`	SD 1.5	Versatile artistic model
`realistic-vision-v6`	SD 1.5	Portrait specialist

FLUX Pipeline Variants

Model	Pipeline	Use Case
`flux.1-kontext-dev`	FluxKontextPipeline	Instruction-based image editing — pass an input image + edit prompt (added v2.0.14)
`flux.1-fill-dev`	FluxFillPipeline	Inpainting / outpainting
`flux.1-canny-dev`	FluxControlPipeline	Canny edge control
`flux.1-depth-dev`	FluxControlPipeline	Depth map control
`qwen-image-edit-2511`	QwenImageEditPlusPipeline	Qwen's current edit model — one to three reference images, better character and group-photo consistency than 2509. bf16, 58 GB: a 64 GB+ Mac or CPU offload on CUDA. Apache 2.0

Apache-2.0 Commercial-Friendly

Model	Origin	Params	Notes
`chroma1-hd`	lodestones	8.9B	FLUX-schnell derivative with custom MMDiT masking + 250M timestep FFN (added v2.0.14)
`flux.1-schnell`	Black Forest Labs	12B	4-step distilled
`flux.2-klein-4b`	Black Forest Labs	4B	FLUX.2 family, MPS-friendly
`z-image-turbo`	Alibaba (Tongyi)	6B	8-step DMD
`sana-1.5`	NVIDIA	1.6B	Fastest >1024² generation
`cogview4`	Zhipu AI	6B	Multilingual including CJK
`pixart-sigma`	PixArt	0.6B	Fits 6GB GPUs
`lumina-2`	Alpha-VLLM	2B	Open multimodal foundation
`auraflow`	Fal	6.8B	Latest open MMDiT
`omnigen`	BAAI	3.8B	Unified gen + edit

💾 GGUF Models - Reduced Memory Requirements

GGUF quantized models enable running FLUX.1-dev on budget hardware:

GGUF Variant	VRAM	Quality	Best For
`flux.1-dev-gguf-q4ks`	6GB	⭐⭐⭐⭐	Recommended - RTX 3060/4060
`flux.1-dev-gguf-q3ks`	4GB	⭐⭐⭐	Mobile GPUs, GTX 1660 Ti
`flux.1-dev-gguf-q2k`	3GB	⭐⭐	Entry-level hardware
`flux.1-dev-gguf-q6k`	10GB	⭐⭐⭐⭐⭐	RTX 3080/4070+

📖 Complete GGUF Guide - Hardware recommendations, installation, and optimization tips

🍎 MLX Models — Apple Silicon native

MLX entries run through mflux on Apple Silicon (M1/M2/M3/M4). On M-series hardware they are typically 2-3× faster than the same model on the PyTorch + MPS path. Install with pip install 'ollamadiffuser[mlx]'.

Text-to-image:

Entry	Family	Quant	Disk	Recommended VRAM	License
`flux.1-schnell-mlx`	FLUX.1	Q8	14 GB	16 GB (M1 32GB)	Apache 2.0
`flux.1-schnell-mlx-q4`	FLUX.1	Q4	8 GB	12 GB (M4 16GB)	Apache 2.0
`flux.1-dev-mlx`	FLUX.1	Q8	14 GB	16 GB	Non-Commercial
`flux.2-klein-4b-mlx`	FLUX.2 Klein	Q8	7 GB	12 GB (M4 16GB)	Apache 2.0
`flux.2-klein-9b-mlx`	FLUX.2 Klein	Q8	13 GB	20 GB	Non-Commercial (gated)
`flux.2-klein-base-4b-mlx`	FLUX.2 Klein base (undistilled, 50-step CFG)	Q8	7 GB	12 GB	Apache 2.0
`flux.1-krea-dev-mlx`	FLUX.1 Krea (12B)	Q8	14 GB	16 GB	Non-Commercial (gated)
`z-image-turbo-mlx`	Z-Image (6B, 8-step DMD)	Q8	8 GB	12 GB (M4 16GB)	Apache 2.0
`z-image-mlx`	Z-Image base (6B, 50-step CFG)	Q8	8 GB	12 GB	Apache 2.0
`z-anime-mlx`	Z-Anime (anime fine-tune of Z-Image base)	Q8	8 GB	12 GB	Apache 2.0
`qwen-image-mlx`	Qwen-Image (20B)	Q8	22 GB	24 GB	Apache 2.0
`qwen-image-2.1-mlx`	Qwen-Image-2.1 (mflux 0.20+)	Q8	34 GB	64 GB (measured 45.8 GB peak)	Qwen Research (non-commercial)
`qwen-image-2.1-turbo-mlx`	Qwen-Image-2.1-Turbo (official 8-step, text-to-image)	Q8	33 GB	64 GB (measured 38.7 GB peak)	Qwen Research (non-commercial)
`ming-image-design-mlx`	Ming-Image-0.1-Design (6B, RGBA, 12 steps)	Q8	53 GB	64 GB	MIT
`boogu-image-turbo-mlx`	Boogu Image (10B, 4-step DMD)	Q8	12 GB	16 GB	Apache 2.0
`ernie-image-turbo-mlx`	ERNIE-Image (8B, 8-step)	Q8	10 GB	14 GB	Apache 2.0
`ernie-image-mlx`	ERNIE-Image SFT (8B, 50-step CFG)	Q8	10 GB	14 GB	Apache 2.0
`krea-2-turbo-mlx`	Krea 2 (12B, 8-step)	Q8	14 GB	20 GB	Krea 2 Community (gated)
`krea-2-raw-mlx`	Krea 2 Raw (12B base, for fine-tuning)	Q8	14 GB	20 GB	Krea 2 Community (gated)
`lens-turbo-mlx`	Lens (3.8B + 20B text encoder, 4-step)	Q8	16 GB	20 GB	Other
`ideogram-4-mlx`	Ideogram 4 (9B, preset schedules)	Q8	12 GB	18 GB	Other
`fibo-mlx`	FIBO (8B, JSON prompts)	Q8	11 GB	16 GB	Bria (gated)

Image editing / control:

Entry	Required inputs	License
`flux.1-kontext-dev-mlx`	`image=`	Non-Commercial
`flux.1-fill-dev-mlx`	`image=`, `mask_image=`	Non-Commercial
`flux.1-redux-dev-mlx`	`redux_images=[...]`	Non-Commercial
`flux.1-depth-dev-mlx`	`image=`	Non-Commercial
`flux.1-controlnet-canny-mlx`	`control_image=` (canny edges)	Non-Commercial
`flux.1-controlnet-upscaler-mlx`	`control_image=` (low-res source)	Non-Commercial
`qwen-image-edit-mlx`	`image=` (2509)	Apache 2.0
`qwen-image-edit-2511-mlx`	`image=`, more as `images=`	Apache 2.0
`qwen-image-2.1-edit-mlx`	`image=` (up to ten with `images=`), optional `mask_image=`, `strength=`	Qwen Research (non-commercial)
`qwen-image-2.1-controlnet-mlx`	`control_image=`, or `image=` + `mask_image=` to inpaint	Qwen Research (non-commercial)
`flux.2-klein-9b-kv-edit-mlx`	`image=` (one or more references)	Non-Commercial (gated)
`fibo-edit-mlx`	`image=`	Bria (gated)
`seedvr2-3b-mlx`	`image=` (upscales it)	Apache 2.0
`seedvr2-7b-mlx`	`image=` (upscales it)	Apache 2.0
`z-image-turbo-controlnet-mlx`	`control_image=`, `control_type=` (canny / depth / pose / hed / mlsd)	Apache 2.0

🎬 Video — LTX-2 on Apple Silicon

Video entries (model_type: ltx-video-mlx) run LTX-2 through ltx-2-mlx, a pure-MLX port: text-to-video with 48 kHz stereo audio, image-to-video, audio-to-video, and LTX-2.5's predicted durations. This is the one strategy that shells out rather than importing — ltx-2-mlx is a three-package monorepo installed with uv sync that manages its own weight packs, and its CLI is the surface its author supports.

Experimental. Both the runtime and the weight packs (dgrauet/ltx-2.*-mlx) are one author's community port, not Lightricks' upstream release, and they are young — expect the CLI and pack layout to move. If a pack is renamed or withdrawn these entries stop pulling; please open an issue if that happens.

Entry	Pack	Disk	RAM	Default mode
`ltx-2.3-mlx-q4`	int4	12 GB	16 GB+	distilled, low-ram
`ltx-2.3-mlx-q8`	int8	21 GB	32 GB+	two-stage
`ltx-2.3-mlx-bf16`	bf16	42 GB	64 GB+	two-stages-hq, 720p
`ltx-2.5-mlx-q4` / `-q8` / `-bf16`	2.5 packs (gated)	13 / 22 / 44 GB	16 / 32 / 64 GB+	predicted duration

# One-time: the runtime it drives
git clone https://github.com/dgrauet/ltx-2-mlx && cd ltx-2-mlx && uv sync --all-extras
# then put .venv/bin on PATH, or export LTX2MLX_BIN=/path/to/ltx-2-mlx

ollamadiffuser pull ltx-2.3-mlx-q8
ollamadiffuser run ltx-2.3-mlx-q8

# Text to video (4s at 24fps). Frame counts are 8k+1; `seconds` rounds for you.
curl -X POST http://localhost:8000/api/generate/video \
  -F prompt="a courtyard in the rain, slow pan" -F seconds=4 -o clip.mp4

# Image to video — animate a still
curl -X POST http://localhost:8000/api/generate/video \
  -F prompt="she turns and smiles" -F image=@portrait.png -F seconds=4 -o clip.mp4

# Audio to video — a voice drives the face
curl -X POST http://localhost:8000/api/generate/video \
  -F prompt="a woman speaking to camera" -F audio=@line.wav -o clip.mp4

Video models refuse the image endpoints rather than returning one frame of the clip, and /api/generate/video answers 400 (not 500) for the things a caller can fix: no model loaded, an image model loaded, ltx-2-mlx not installed, a frame count off the grid.

🎬 Video on NVIDIA — Wan and LTX-2 through diffusers

Off Apple Silicon, video goes through diffusers' own pipelines (model_type: diffusers-video). Same endpoint and the same mp4 back: POST /api/generate/video with a prompt, and optionally image (first frame), frames or seconds, width, height, steps, cfg_scale, seed.

Entry	Model	Text / image to video	Default	Disk	VRAM	License
`wan2.1-t2v-1.3b`	Wan 2.1 T2V 1.3B	text	480P, 16 fps, 5 s	29 GB	8 GB+	Apache 2.0
`wan2.2-ti2v-5b`	Wan 2.2 TI2V 5B	both	720P (1280×704), 24 fps, 5 s	35 GB	24 GB+	Apache 2.0
`ltx-2.3-distilled`	LTX-2.3 distilled, with sound	both	768×512, 24 fps, 8 steps	95 GB	24 GB+ (offload)	LTX-2 Community

LTX-2 writes its soundtrack into the mp4. The LTX-2 diffusers entries refuse the Apple GPU — diffusers' LTX-2 uses float64, which Metal lacks — and point to the ltx-*-mlx entries instead.

Hardware fit at a glance:

Mac Mini M4 16 GB can run anything marked ✅ above (Q4 FLUX.1-schnell, FLUX.2 Klein 4B, Z-Image-Turbo).
Mac Pro M1 32 GB / M2 Pro 32 GB+ can run all entries except the 20B Qwen-Image at the larger resolutions.

Quick start:

pip install 'ollamadiffuser[mlx]'
ollamadiffuser pull z-image-turbo-mlx    # smallest Apache-2.0 option
ollamadiffuser run z-image-turbo-mlx

🎛️ ControlNet Features

⚡ Lazy Loading Architecture

New in v1.1.0: ControlNet preprocessors use intelligent lazy loading:

Instant Startup: ollamadiffuser --help runs immediately without downloading models
On-Demand Loading: Preprocessors initialize only when actually needed
Automatic Initialization: Seamless loading when uploading control images
User Control: Manual initialization available for pre-loading

Available Control Types

Canny Edge Detection: Structural control with edge maps
Depth Estimation: 3D structure control with depth maps
OpenPose: Human pose and body position control
Scribble/Sketch: Artistic control with hand-drawn inputs
Advanced Types: HED, MLSD, Normal, Lineart, Anime Lineart, Content Shuffle

ControlNet Models

# SD 1.5 ControlNet Models
ollamadiffuser pull controlnet-canny-sd15
ollamadiffuser pull controlnet-depth-sd15
ollamadiffuser pull controlnet-openpose-sd15
ollamadiffuser pull controlnet-scribble-sd15

# SDXL ControlNet Models
ollamadiffuser pull controlnet-canny-sdxl
ollamadiffuser pull controlnet-depth-sdxl

🔄 LoRA Support

Dynamic LoRA Management

# Download LoRA from Hugging Face
ollamadiffuser lora pull "openfree/flux-chatgpt-ghibli-lora"

# Load LoRA with custom strength
ollamadiffuser lora load ghibli --scale 1.2

# Unload LoRA
ollamadiffuser lora unload

LoRAs on Apple Silicon (MLX models)

The same commands work on -mlx models. mflux takes a LoRA when the model is built, so loading or unloading one rebuilds the model with it baked in, and costs nothing per step afterwards. Measured on an M3 Ultra with FLUX.2 klein 4B: 45 s to load the LoRA the first time, most of it the 325 MB download; 0.3 s to unload it. LoRAs stack: load a speed LoRA and a style LoRA and both apply. The rebuild releases the old weights before it reads the new ones, and refuses — with a sentence saying how much it needs — when the Mac has no room for one more copy.

ollamadiffuser lora pull Limbicnation/pixel-art-lora -w pytorch_lora_weights.safetensors -a pixel
ollamadiffuser lora load pixel --scale 1.0

Families that take LoRAs: FLUX.1 (and Kontext), FLUX.2 klein (and its edit variant), Z-Image, Qwen-Image, Qwen-Image-2.1 (generation, edit and ControlNet), Krea 2, ERNIE-Image. Boogu, Lens, Ideogram 4, FIBO, Ming-Image and SeedVR2 don't in mflux yet, and say so.

Qwen-Image-2.1 in 6 steps — Viggle's turbo LoRA with mflux's matching scheduler, for generation and editing alike. Measured on an M3 Ultra at 1024×1024: about 27 s per image (37 s for the first) against 110 s for 25 steps without it, with the sign's Chinese text still right. The Qwen-Image-2.1 entries keep LoRAs as live adapters (bake_lora: false): baked into the 8-bit weights, this one lost more than half its effect.

ollamadiffuser lora pull Viggle/Qwen-Image-2.1-viggle-turbo \
  -w Qwen-Image-2.1-viggle-turbo-v0.3-6step-lora-r256.safetensors -a qwen21-turbo
ollamadiffuser lora load qwen21-turbo
# then generate with 6 steps and the matching scheduler:
curl -s localhost:8000/api/generate -H 'Content-Type: application/json' \
  -d '{"prompt": "a cabin by the sea", "steps": 6, "scheduler": "viggle_turbo"}' -o cabin.png

On LTX video, pass lora= (a local .safetensors or a hub repo id) and lora_strength= to /api/generate/video for a style or motion LoRA; with a control video, lora is the IC-LoRA.

Web UI LoRA Integration

Easy Download: Enter Hugging Face repository ID
Strength Control: Adjust LoRA influence with sliders
Real-time Loading: Load/unload LoRAs without restarting
Alias Support: Create custom names for your LoRAs

🌐 Multiple Interfaces

Command Line Interface

# One picture, one edit, one clip: each prints the file it wrote
ollamadiffuser generate "a red fox in snow" -m z-image-turbo-mlx -o fox.png
ollamadiffuser edit "make it a snowy night" -i cabin.png -m qwen-image-2.1-edit-mlx
ollamadiffuser video "waves at dusk" -m ltx-2.3-mlx-q8 -s 4

# Pull and run a model as a server
ollamadiffuser pull stable-diffusion-1.5
ollamadiffuser run stable-diffusion-1.5

# What is running, and stopping it
ollamadiffuser ps                     # every server on this machine and its model
ollamadiffuser stop --port 8010       # or --all

# Model registry management
ollamadiffuser registry list
ollamadiffuser registry list --installed-only
ollamadiffuser registry check-gguf

# Configuration management
ollamadiffuser config                                    # show all config
ollamadiffuser config set models_dir /mnt/ssd/models     # custom model path
ollamadiffuser config set server.port 9000               # change server port

# In another terminal, generate images via API
curl -X POST http://localhost:8000/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "a futuristic cityscape",
    "negative_prompt": "blurry, low quality",
    "num_inference_steps": 30,
    "guidance_scale": 7.5,
    "width": 1024,
    "height": 1024
  }' \
  --output image.png

Web UI

# Start web interface
ollamadiffuser --mode ui
Open http://localhost:8001

Features:

Responsive Design: Works on desktop and mobile
Real-time Status: Model and LoRA loading indicators
ControlNet Integration: File upload with preprocessing
Parameter Controls: Intuitive sliders and inputs

REST API

# Start API server
ollamadiffuser --mode api
ollamadiffuser load stable-diffusion-1.5

# Text-to-image
curl -X POST http://localhost:8000/api/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "a beautiful landscape", "width": 1024, "height": 1024, "seed": 42}'

# Image-to-image
curl -X POST http://localhost:8000/api/generate/img2img \
  -F "prompt=oil painting style" \
  -F "strength=0.75" \
  -F "image=@input.png" \
  --output result.png

# Inpainting
curl -X POST http://localhost:8000/api/generate/inpaint \
  -F "prompt=a red car" \
  -F "image=@photo.png" \
  -F "mask=@mask.png" \
  --output inpainted.png

# API docs: http://localhost:8000/docs

MCP Server (AI Assistant Integration)

OllamaDiffuser includes a Model Context Protocol server for integration with AI assistants like OpenClaw, Claude Code, and Cursor.

# Install MCP support
pip install "ollamadiffuser[mcp]"

# Start MCP server (stdio transport)
ollamadiffuser mcp

MCP client configuration (e.g. claude_desktop_config.json):

{
  "mcpServers": {
    "ollamadiffuser": {
      "command": "ollamadiffuser-mcp"
    }
  }
}

Works with mcp 1.x and 2.x.

For Claude Code: claude mcp add ollamadiffuser -- ollamadiffuser mcp.

Available MCP tools:

generate_image -- Generate images from text prompts (auto-loads model; takes a scheduler and a lora). Returns the saved file's path as text and the image itself; when an ollamadiffuser run server already holds the model, that server does the work
edit_image -- Edit a picture by instruction: the picture's path, more references, a mask, a strength
generate_video -- Make a short video, from a prompt, a first frame or an audio track; returns the mp4's path
list_servers -- The OllamaDiffuser servers running on this machine and the model each holds
list_models -- List available and installed models
load_model -- Load a model into memory (says why when it won't fit)
get_status -- Check device, loaded model, and system status

Using OllamaDiffuser from agents

An agent with a shell (Claude Code, Codex, OpenClaw, a script) needs no server and no MCP:

ollamadiffuser generate "a poster that reads 海边小屋" -m qwen-image-2.1-turbo-mlx -o poster.png --json
# {"path": "/…/poster.png", "kind": "image", "model": "qwen-image-2.1-turbo-mlx", "seed": 1873, "elapsed_s": 36.2, "server": null}

One command, one file. generate, edit and video print the file they wrote. With --json, stdout is one JSON object, or {"error": …} with exit code 1, and everything else goes to stderr.
No second copy. When a server already holds the model (ollamadiffuser run, the ComfyUI nodes, another agent), the job goes to it. Otherwise the model loads for the job and is let go after it.
Seeing what runs. ollamadiffuser ps --json lists every server on this machine and its model, including servers started by older versions or from another config directory. list --json lists installed models with their memory needs, and stop --port N / --all stops servers.
Memory. On Apple Silicon, where the GPU shares the machine's memory, a model is not loaded when less than its minimum is free. The error gives the numbers. --force (or OLLAMADIFFUSER_SKIP_MEMORY_CHECK=1) loads it anyway.
A ready-made skill for Claude Code and other skill-aware agents is at integrations/claude-code/ollamadiffuser/SKILL.md. Copy the folder to ~/.claude/skills/.

OpenClaw AgentSkill

An OpenClaw skill is included at integrations/openclaw/SKILL.md. It uses the REST API with response_format=b64_json for agent-friendly base64 image responses. Copy the skill directory to your OpenClaw skills folder or publish to ClawHub.

ComfyUI Nodes

Three custom nodes at integrations/comfyui/ bring OllamaDiffuser's models into a ComfyUI graph — Text to Image, Edit Image (up to four references) and Video (first frame, audio, a control video to follow, LoRA). ComfyUI can't read the MLX builds, so the nodes don't load anything: they call OllamaDiffuser's HTTP API and hand back an ordinary IMAGE or VIDEO for the rest of the graph. A server already running the model is reused; otherwise one is started, after ComfyUI's own models are unloaded, and stopped again afterwards unless keep_loaded is on.

ln -s /path/to/ollamadiffuser/integrations/comfyui ~/ComfyUI/custom_nodes/comfyui-ollamadiffuser

The nodes live in the GitHub repository, not in the pip package — clone the repo, or download the folder, to install them.

Base64 JSON API Response

For AI agents and messaging platforms, use response_format=b64_json to get images as JSON:

curl -X POST http://localhost:8000/api/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "a sunset over mountains", "response_format": "b64_json"}'

Response: {"image": "<base64 PNG>", "format": "png", "width": 1024, "height": 1024}

Python API

from ollamadiffuser.core.models.manager import model_manager

# Load model
success = model_manager.load_model("stable-diffusion-1.5")
if success:
    engine = model_manager.loaded_model

    # Text-to-image (seed is optional; omit for random)
    image = engine.generate_image(
        prompt="a beautiful sunset",
        width=1024,
        height=1024,
        seed=42,
    )
    image.save("output.jpg")

    # Image-to-image
    from PIL import Image
    input_img = Image.open("photo.jpg")
    result = engine.generate_image(
        prompt="watercolor painting",
        image=input_img,
        strength=0.7,
    )
    result.save("img2img_output.jpg")
else:
    print("Failed to load model")

📦 Model Ecosystem

Base Models

Stable Diffusion 1.5: Classic, reliable, fast (img2img + inpainting)
Stable Diffusion XL: High-resolution, detailed (img2img + inpainting, scheduler overrides)
Stable Diffusion 3.5: Medium, Large, and Large Turbo variants
FLUX.1: schnell, dev, Fill, Canny, Depth pipeline variants
HiDream: Multi-prompt generation with bfloat16
AnimateDiff: Video/animation generation

Next-Generation Models

FLUX.2: 32B dev and 4B Klein variants from Black Forest Labs
Chinese Models: CogView4 (Zhipu), Kolors (Kuaishou), Hunyuan-DiT (Tencent), Z-Image (Alibaba)
Efficient Models: SANA 1.5 (1.6B), PixArt-Sigma (0.6B) -- high quality at low VRAM
Open Models: AuraFlow (6.8B, Apache 2.0), OmniGen (3.8B, MIT), Lumina 2.0 (2B, Apache 2.0)

Fast / Turbo Models

SDXL Turbo: Single-step inference from Stability AI
SDXL Lightning: 4-step single-file checkpoint from ByteDance (6.5 GB download)
Z-Image Turbo: 8-step turbo from Alibaba

Community Fine-Tunes

RealVisXL V4: Photorealistic SDXL, very popular
DreamShaper: Versatile artistic SD 1.5 model
Realistic Vision V6: Portrait specialist

GGUF Quantized Models

FLUX.1-dev GGUF: 7 quantization levels (3GB-16GB VRAM)
Memory Efficient: Run high-quality models on budget hardware
Optional Install: ollamadiffuser enable gguf

ControlNet Models

SD 1.5 ControlNet: 4 control types (canny, depth, openpose, scribble)
SDXL ControlNet: 2 control types (canny, depth)

LoRA Support

Hugging Face Integration: Direct download from HF Hub
Local LoRA Files: Support for local .safetensors files
Dynamic Loading: Load/unload without model restart
Strength Control: Adjustable influence (0.1-2.0)

⚙️ Architecture

Strategy Pattern Engine

Each model type has a dedicated strategy class handling loading and generation:

InferenceEngine (facade)
  -> SD15Strategy            (512x512, float16 on MPS, img2img, inpainting)
  -> SDXLStrategy            (1024x1024, float16 on MPS, diffusers force_upcast, img2img, inpainting, scheduler overrides, single-file)
  -> FluxStrategy            (schnell/dev/Fill/Canny/Depth, bfloat16 on MPS, dynamic pipeline class)
  -> SD3Strategy             (1024x1024, float16 on MPS, 28 steps, guidance=3.5)
  -> ControlNetStrategy      (SD15 + SDXL, float16 on MPS, SDXL uses diffusers force_upcast)
  -> VideoStrategy           (AnimateDiff, float16 on MPS, 16 frames)
  -> HiDreamStrategy         (bfloat16 on MPS, multi-prompt)
  -> GGUFStrategy            (quantized via stable-diffusion-cpp)
  -> GenericPipelineStrategy (any diffusers pipeline via config, per-model dtype on MPS, opt-in VAE upcast)

The GenericPipelineStrategy dynamically loads any diffusers pipeline class specified in the model registry, so new models can be added with zero code changes.

Configuration

Models are automatically configured with optimal settings:

Memory Optimization: Attention slicing, CPU offloading
Device Detection: Automatic CUDA/MPS/CPU selection
Precision Handling: FP16/BF16 per model type
Safety Disabled: Unified SAFETY_DISABLED_KWARGS (no monkey-patching)
Smart Downloads: Pipeline-only filtering by model type — skips ONNX, Flax, root checkpoints, and safety_checker

🔧 Advanced Usage

ControlNet Parameters

# Fine-tune ControlNet behavior
image = engine.generate_image(
    prompt="architectural masterpiece",
    control_image=control_img,
    controlnet_conditioning_scale=1.2,  # Strength (0.0-2.0)
    control_guidance_start=0.0,         # When to start (0.0-1.0)
    control_guidance_end=1.0            # When to end (0.0-1.0)
)

GGUF Model Usage

# Check GGUF support
ollamadiffuser registry check-gguf

# Download GGUF model for your hardware
ollamadiffuser pull flux.1-dev-gguf-q4ks  # 6GB VRAM
ollamadiffuser pull flux.1-dev-gguf-q3ks  # 4GB VRAM

# Use with optimized settings
ollamadiffuser run flux.1-dev-gguf-q4ks

Batch Processing

from ollamadiffuser.core.utils.controlnet_preprocessors import controlnet_preprocessor

# Pre-initialize for faster processing
controlnet_preprocessor.initialize()

# Process multiple images
prompt = "beautiful landscape"  # Define the prompt
for i, image_path in enumerate(image_list):
    control_img = controlnet_preprocessor.preprocess(image_path, "canny")
    result = engine.generate_image(prompt, control_image=control_img)
    result.save(f"output_{i}.jpg")

API Integration

import requests

# Initialize ControlNet preprocessors
response = requests.post("http://localhost:8000/api/controlnet/initialize")

# Check available preprocessors
response = requests.get("http://localhost:8000/api/controlnet/preprocessors")
print(response.json()["available_types"])

# Generate with file upload
with open("control.jpg", "rb") as f:
    response = requests.post(
        "http://localhost:8000/api/generate/controlnet",
        data={"prompt": "beautiful landscape"},
        files={"control_image": f}
    )

📚 Documentation & Guides

GGUF Models Guide: Complete guide to memory-efficient GGUF models
ControlNet Guide: Comprehensive ControlNet usage and examples
Website Documentation: Complete tutorials and guides

🚀 Performance & Hardware

Minimum Requirements

RAM: 8GB system RAM
Storage: 10GB free space
Python: 3.10+

Recommended Hardware

For Regular Models

GPU: 8GB+ VRAM (NVIDIA/AMD)
RAM: 16GB+ system RAM
Storage: SSD with 50GB+ free space

For Apple Silicon (Mac Mini / MacBook)

16GB unified memory: SANA 1.5, Lumina 2.0, DreamShaper, SD 1.5, SDXL/SDXL Turbo, GGUF q2k-q5ks
24GB+ unified memory: CogView4, Hunyuan-DiT, FLUX.1-schnell, GGUF q6k-q8
32GB unified memory: Kolors, SD 3.5 Large, all MPS-supported models
GGUF with Metal: ollamadiffuser enable gguf (sets CMAKE_ARGS=-DSD_METAL=ON for GPU acceleration automatically)
Note: CPU offload does not help on Apple Silicon (unified memory) -- the full model must fit in RAM
Run ollamadiffuser recommend to see what fits your hardware

For GGUF Models (Memory Efficient)

GPU: 3GB+ VRAM (or CPU only)
RAM: 8GB+ system RAM (16GB+ for CPU inference)
Storage: SSD with 20GB+ free space

Supported Platforms

CUDA: NVIDIA GPUs (recommended)
MPS: Apple Silicon (M1/M2/M3/M4) -- native support for 30+ models including GGUF
CPU: All platforms (slower but functional)

🔧 Troubleshooting

Installation Issues

Missing Dependencies (cv2/OpenCV Error)

The default install already includes OpenCV (it ships as a prebuilt wheel), so this is rare. If you somehow hit ModuleNotFoundError: No module named 'cv2':

# Verify and repair the install
ollamadiffuser verify-deps

# Or install OpenCV directly
pip install "opencv-python>=4.8.0"

GGUF Support Issues

# Enable the GGUF backend (compiles once; sets build flags for you)
ollamadiffuser enable gguf

# Check GGUF support
ollamadiffuser registry check-gguf

# See GGUF_GUIDE.md for detailed troubleshooting

Complete Dependency Check

# Run comprehensive system diagnostics
ollamadiffuser doctor

# Verify and install missing dependencies interactively
ollamadiffuser verify-deps

Clean Installation

If you're having persistent issues, the isolated curl | sh installer sidesteps any conflicts in your existing Python environment:

# Fully isolated reinstall (never touches your system Python)
rm -rf ~/.ollamadiffuser
curl -fsSL https://raw.githubusercontent.com/LocalKinAI/ollamadiffuser/main/install.sh | sh

Or, staying in your own environment:

pip uninstall ollamadiffuser
pip install --no-cache-dir ollamadiffuser   # compile-free core
ollamadiffuser enable gguf   # add optional backends only if you need them
ollamadiffuser verify-deps

Common Issues

Slow Startup

If you experience slow startup, ensure you're using the latest version with lazy loading:

git pull origin main
pip install -e .

ControlNet Not Working

# Check preprocessor status
python -c "
from ollamadiffuser.core.utils.controlnet_preprocessors import controlnet_preprocessor
print('Available:', controlnet_preprocessor.is_available())
print('Initialized:', controlnet_preprocessor.is_initialized())
"

# Manual initialization
curl -X POST http://localhost:8000/api/controlnet/initialize

Memory Issues

# Use GGUF models for lower memory usage
ollamadiffuser pull flux.1-dev-gguf-q4ks  # 6GB VRAM
ollamadiffuser pull flux.1-dev-gguf-q3ks  # 4GB VRAM

# Use smaller image sizes via API
curl -X POST http://localhost:8000/api/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "test", "width": 512, "height": 512}' \
  --output test.png

# CPU offloading is automatic
# Close other applications to free memory
# Use basic preprocessors instead of advanced ones

# MLX models (Apple Silicon) hold MLX's buffer cache to 2 GB; on a Mac with
# memory to spare, a larger bound (in GB) lets MLX reuse more of it
OLLAMADIFFUSER_MLX_CACHE_LIMIT_GB=8 ollamadiffuser run qwen-image-2.1-turbo-mlx

Platform-Specific Issues

macOS Apple Silicon

# If you encounter OpenCV issues on Apple Silicon
pip uninstall opencv-python
pip install "opencv-python-headless>=4.8.0"

# GGUF with Metal acceleration — `enable gguf` sets CMAKE_ARGS=-DSD_METAL=ON for you
ollamadiffuser enable gguf

Windows

# If you encounter build errors
pip install --only-binary=all "opencv-python>=4.8.0"

# GGUF on CPU: `ollamadiffuser enable gguf`
# GGUF with CUDA acceleration (advanced — set the flag yourself):
CMAKE_ARGS="-DSD_CUDA=ON" pip install stable-diffusion-cpp-python

Linux

# If you need system dependencies
sudo apt-get update
sudo apt-get install libgl1-mesa-glx libglib2.0-0
pip install opencv-python>=4.8.0

Debug Mode

# Enable verbose logging
ollamadiffuser --verbose run model-name

🤝 Contributing

We welcome contributions! Please check the GitHub repository for contribution guidelines.

🤝 Community & Support

Quick Actions

🐛 Report a Bug - Found an issue? Let us know
💡 Feature Request - Have an idea? Share it with us
💬 Join Discussions - Community discussion
⭐ Star on GitHub - Show your support

Community Driven

OllamaDiffuser is an open-source project that thrives on community feedback. Every suggestion, bug report, and contribution helps make it better for everyone.

Open Source • Community Driven • Actively Maintained

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

Stability AI: For Stable Diffusion models
Black Forest Labs: For FLUX.1 and FLUX.2 models
Alibaba (Tongyi-MAI): For Z-Image Turbo
NVIDIA (Efficient-Large-Model): For SANA 1.5
Zhipu AI (THUDM): For CogView4
Kuaishou (Kwai-Kolors): For Kolors
Tencent (Hunyuan): For Hunyuan-DiT
Alpha-VLLM: For Lumina 2.0
PixArt-alpha: For PixArt-Sigma
Fal: For AuraFlow
BAAI (Shitao): For OmniGen
ByteDance: For SDXL Lightning
city96: For FLUX.1-dev GGUF quantizations
Hugging Face: For model hosting and diffusers library
Anthropic: For Model Context Protocol (MCP)
OpenClaw: For AI agent ecosystem integration
ControlNet Team: For ControlNet architecture
Community: For feedback and contributions

📞 Support

Issues: GitHub Issues
Discussions: GitHub Discussions

Ready to get started? Install from PyPI: pip install ollamadiffuser or visit ollamadiffuser.com 🎨✨

Metadata

Release files for ollamadiffuser 2.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ollamadiffuser 2.3.0
File	Size	Uploaded
ollamadiffuser-2.3.0.tar.gz	344.5 kB	Oct 11, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for ollamadiffuser 2.3.0
File	Interpreter	ABI	Platform	Reset
ollamadiffuser-2.3.0-py3-none-any.whl 244.5 kB Oct 11, 2026	Python 3	none	any	Details

Total release size: 589.0 kB

Release files / ollamadiffuser-2.3.0.tar.gz

Download URL	ollamadiffuser-2.3.0.tar.gz
Size	344.5 kB
Tags	Source
SHA-256 checksum How to use checksums	`2ae82d542bb9ec69f257f62c08677cd44e9dd14ba5eb3d8c566c6db5b72d318d`
BLAKE2b-256 checksum How to use checksums	`474a2806c119b6068771d5e62885393c3d6c789b027dda80d71a9687d1618888`
Upload date	Oct 11, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.10.0`

Release files / ollamadiffuser-2.3.0-py3-none-any.whl

Download URL	ollamadiffuser-2.3.0-py3-none-any.whl
Size	244.5 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`072c7bfb5e9cc6752f397d7054376168448a24fe7b71b11925bed07107d2a882`
BLAKE2b-256 checksum How to use checksums	`bcc729697b721165a3be3218258b35dc66e9dec12c4759a10bdfb8f760afff86`
Upload date	Oct 11, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.10.0`