aircanvas 0.1.0

Run 20B+ image & video generation models on 4–8 GB GPUs.

Quickstart | Configurations | Supported Models | Benchmarks | FAQ

Measured, not promised: a 20.4B Qwen-Image generating 1024² images on a 6 GB laptop RTX 4050 — a model that needs a 40 GB+ card by any other means.

AirCanvas is AirLLM for diffusion models: it splits a model's transformer into per-block shards on disk (fp8/NF4-quantized at split time), then streams each block to the GPU just-in-time during the denoise loop. VRAM scales with the largest block, not the model; system RAM holds only a small pinned prefetch ring.

Why the economics work where AirLLM's don't: an LLM re-reads the whole model per token; a diffusion model runs 20–50 forward passes total — and video steps are so compute-heavy the transfer vanishes entirely.

Updates

• [2026/08] v0.1.0 — first working release. FLUX.1, Qwen-Image, SD3/3.5, Wan 2.1/2.2 (dual-expert handover), HunyuanVideo, CogVideoX adapters; fp8 and NF4 shard formats; automatic VRAM/RAM/disk budget solver; aircanvas doctor.
• [2026/08] Wan 2.1 14B verified end-to-end on a 6 GB GPU — 562 GB of weights streamed through a single overnight run (docs/BENCHMARKS.md).
• [2026/08] Video streaming measured effectively free: Wan 2.1 at 480×832×81 frames runs at 1.02× the full-VRAM step time with byte-identical output.

Star History

Table of Contents

• Quickstart
• How it works
• Model compression: fp8 & NF4 shards
• Configurations
• Supported Models
• Benchmarks
• FAQ
• Acknowledgement
• Citing AirCanvas
• Contribution

Quickstart

1. Install

pip install aircanvas            # from PyPI
pip install "aircanvas[nf4]"     # + bitsandbytes for NF4 shards (CUDA required)
pip install -e .[dev]            # from a clone, for contributors

Windows and Linux; Python ≥ 3.10; torch ≥ 2.4; diffusers ≥ 0.35. Developed on a 6 GB / 16 GB Windows laptop — the low-resource path is the tested path, including the failure modes (commit-pressure mmap fallbacks, resumable everything).

2. Generate

from aircanvas import AirPipeline

pipe = AirPipeline.from_pretrained(
    "Qwen/Qwen-Image",  # 20B — split to nf4 shards on first use
    compression="nf4",
)
image = pipe("a watercolor fox reading a newspaper", num_inference_steps=20).images[0]
print(pipe.report())  # phase times, IO breakdown, residency plan

The first run downloads the transformer, splits it into a persistent per-block shard cache (outside the repo, under your HF cache), and streams from there forever after — the original checkpoint can be deleted.

3. Or use the CLI

aircanvas doctor                                    # what can THIS box run?
aircanvas split Qwen/Qwen-Image --compression nf4   # one-time reshard
aircanvas run Qwen/Qwen-Image -p "a watercolor fox" # generate

How it works

1. Split once — the DiT is resharded into per-block safetensors (fp8 or NF4 payloads, quality-sensitive tensors kept high-precision), verified against the reader's own parser, resumable, and reusable forever.
2. Stream at runtime — the model is built on the meta device (zero memory); a prefetch worker walks the recorded block schedule: disk → pinned ring → GPU slot pool over a dedicated CUDA stream. No allocations in the hot loop. Streamed output is bitwise-equal to full-VRAM execution (uncompressed) — enforced by tests and demonstrated on real video.
3. Phase-aware placement — text encoders run once and are evicted (embedding cache makes repeats free; encoders too big to LOAD on a small machine are themselves split and streamed), the DiT streams every step, the VAE decodes tiled. A budget solver turns free VRAM/RAM + measured disk bandwidth into the plan, printed with every OOM and report.
4. Cache hierarchy — leftover VRAM pins blocks resident; leftover RAM promotes shards after first read (disk touched once per run); NVMe serves the rest. Wan 2.2's dual experts hand over engines at the boundary timestep so pool memory is never doubled.

Full design docs: docs/ARCHITECTURE.md · docs/RESEARCH.md.

Model compression: fp8 & NF4 shards

Compression is applied once, at split time — runtime just streams smaller files, so disk traffic (usually the bottleneck for images) drops with it:

On the 6 GB dev machine, fp8 shards cut streamed step time from ~300 ms to 150–172 ms on the benchmark model — from IO-bound to compute-bound. NF4 dequantization runs through bitsandbytes' kernel straight into a pre-allocated pool and is tested bitwise-identical to the reference path.

Configurations

Everything routes through AirPipeline.from_pretrained(model_id, ...):

pipe.report() prints per-phase wall time, the IO/prefetch breakdown, and the residency plan the solver chose. aircanvas doctor runs the same solver against your hardware and prints a table of what your box can run.

Supported Models

Benchmarks

All numbers measured on the low-end dev machine (RTX 4050 Laptop 6 GB, 16 GB RAM, NVMe) — details and methodology in docs/BENCHMARKS.md:

FAQ

1. Is it slower than running on a big GPU?

Yes — that's the honest trade. On a 6 GB card AirCanvas is slower than a 24 GB card running natively: minutes per image, near-parity for video (video steps are so compute-heavy the streaming hides completely). Its promise is different: the model runs at all, correctly, with the bottleneck measured and printed by pipe.report().

2. I have a big GPU (16–24 GB+) — is AirCanvas still useful?

Yes, and nothing about it is "small-GPU mode." The budget solver spends whatever VRAM it finds: on a large card it pins most or all blocks permanently resident, streaming quietly shrinks toward zero, and step times converge to native full-VRAM execution — same one-line API. You can also drop compression entirely (compression=None, bitwise-identical output) and raise resolution/frame counts, since activations get the leftover room.

The deeper reason: VRAM only ever needs to hold the largest block (~0.3–1 GB even for 20B+ models), not the model. So when models outgrow your card again — and they will — the same command keeps working. A 40B video DiT that fits no consumer card would stream on a 24 GB card the same way 20B streams on 6 GB today. Run aircanvas doctor to see exactly what your card would pin resident vs stream for each supported model.

3. Does streaming change the output?

No. Streamed output is bitwise-equal to full-VRAM execution with compression=None — enforced by the test suite and demonstrated byte-identical on real video. fp8/NF4 introduce only the usual quantization tolerance, applied once at split time.

4. Where do the shards go, and how big are they?

In a persistent cache under your HF cache home (override with shard_cache=), never inside the repo. Size ≈ the transformer at the chosen precision (roughly ½ for fp8, ¼ for NF4). After splitting you can delete the original checkpoint.

5. I got an out-of-memory error

InsufficientVRAMError prints the exact plan the solver attempted. The usual ladder: enable compression="fp8" (or "nf4"), lower resolution/frames, or set an explicit vram_budget below what other apps are using.

6. NF4 says it requires CUDA

Correct — NF4 shards use bitsandbytes' dequantization kernels, which are CUDA-only. This is enforced with a clear error at split and load. Use "fp8" on non-CUDA setups.

7. Do I need an NVMe drive?

Strongly recommended. The budget solver measures your disk's real bandwidth and warns when a SATA drive (or a distilled few-step model, which hides IO less) will make streaming the bottleneck.

8. How do I use gated models like FLUX.1-dev or SD3.5?

First accept the model's license on its Hugging Face page (a token alone still gets a 403), then authenticate the machine either way:

hf auth login    # one-time; picked up automatically (older hub: huggingface-cli login)
aircanvas run black-forest-labs/FLUX.1-dev -p "..." --hf-token hf_xxx  # or explicit

In Python, pass hf_token="hf_..." to AirPipeline.from_pretrained (or rely on the same ambient login). The token is only needed for the first download: after the one-time split the shard cache is self-contained, so the original checkpoint can be deleted and generation runs fully offline.

9. Does it work on Windows?

Windows is the primary development machine — OneDrive-safe paths, no-mmap-under-commit-pressure fallbacks, and cross-platform memory probing are all part of the tested path.

Acknowledgement

• AirLLM — the layered-inference idea this project adapts to diffusion, and the namesake.
• diffusers — AirCanvas wraps stock pipelines and owns only device/memory placement.
• safetensors, bitsandbytes, accelerate.

Citing AirCanvas

@software{aircanvas,
  author = {Hulley, Kyle},
  title  = {AirCanvas: Run 20B+ image and video diffusion models on 4--8 GB GPUs},
  url    = {https://github.com/kyleADHD/AirCanvas},
  year   = {2026}
}

Contribution

Contributions are welcome — see CONTRIBUTING.md for the dev setup, the test tiers, and the hard rules (bitwise-equivalence gate, no allocations in the hot loop) every PR must keep. Security reports go through SECURITY.md, never public issues.

License

Apache-2.0.