SageAttention for AMD GPUs (ROCm)
This is a ROCm port of SageAttention (thu-ml), the 8-bit
"quantize Q·K^T to INT8" attention that speeds up diffusion and video models as a drop-in replacement
for FlashAttention / SDPA. The rocm branch adds an AMD backend under sageattention/rocm/ and leaves
the public API unchanged, so anything that already calls sageattention.sageattn -- ComfyUI's
--use-sage-attention, diffusers' set_attention_backend("sage"), your own code -- runs on Radeon and
Instinct GPUs without modification. The original CUDA code paths are untouched; the upstream README is
kept as README_upstream.md.
| GPU | Q·K^T | P·V | Kernel |
|---|---|---|---|
| RDNA3: Radeon RX 7000 / PRO W7000 (gfx11; tested gfx1100, gfx1101) | INT8 (WMMA) | FP16 | hand-written HIP kernel, compiled at first use with hipRTC (no compiler toolchain needed); Triton fallback |
| CDNA3: Instinct MI300X (gfx942) | INT8 (MFMA) | FP16, or FP8 (e4m3fnuz) opt-in | Triton |
| other AMD GPUs supported by Triton (gfx90a, gfx950, RDNA4, ...) | INT8 | FP16 | Triton (untested) |
Install is pure Python: nothing is compiled at install time, and it works on Linux and native
Windows (AMD's ROCm PyTorch wheels for Windows plus triton-windows).
Validated on:
| GPU | OS | Software |
|---|---|---|
| Radeon RX 7800 XT 16 GB (gfx1101) | Windows 11 | ROCm 7.2.1 PyTorch wheels (torch 2.9.1), triton-windows 3.6.0.post26 |
| Radeon PRO W7900 48 GB (gfx1100) | Ubuntu 24.04 | ROCm 7.2.2 driver, torch 2.10.0+rocm7.0, Triton 3.6 |
| Instinct MI300X (gfx942) | Linux | ROCm 7.2, torch 2.10.0+rocm7.0, Triton 3.6 |
Headline numbers (details and baselines in Results):
- ComfyUI, Wan2.1 T2V 1.3B (832x480, 33 frames, 20 steps): 2.15x faster sampling than ComfyUI's default attention on the RX 7800 XT
(Windows; 7.33 -> 3.40 s/step) and 1.95x on the Radeon PRO W7900 (Linux; 5.16 -> 2.65 s/step), with
--use-sage-attentionand no other change. - Kernel: on RDNA3 the native kernel runs at 53-56 TFLOP/s on the RX 7800 XT, 1.2-1.4x nunchaku-rocm's
FP16 Triton flash attention and 3.5x PyTorch SDPA on Windows (FLUX 1024² shape, SDPA with
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1; 12x without); on MI300X it is on par with or ahead of aiter's bf16 FlashAttention (CK/assembly) and slower than aiter's FP8 kernel. - diffusers, Wan2.1 T2V 1.3B: 1.21x (33 frames) to 1.25x (81 frames) end to end on the RX 7800 XT against the Triton flash attention, 1.51x against SDPA on MI300X.
- Accuracy: output cosine similarity >= 0.9999 against FP32 attention on synthetic DiT-like inputs and
= 0.99993 on real Wan2.1 activations, as upstream.
Contents
- Install: from source · check the install
- ComfyUI
- Python and diffusers
- Results
- Validation and benchmark scripts
- Limitations
- Configuration
- Other SageAttention ports for AMD
- Credits and license
Install
From PyPI, into the Python environment that runs your model or ComfyUI (it must already have a ROCm build of PyTorch, >= 2.6; tested 2.9.1 and 2.10.0):
pip install sageattention-rocm
# ComfyUI Windows portable, from the folder that contains python_embeded:
python_embeded\python.exe -m pip install sageattention-rocm
The package installs as sageattention 2.2.0 (same module and API as upstream), so ComfyUI's
--use-sage-attention and other callers find it as usual. It is pure Python: nothing is compiled at install time,
and it declares no torch dependency, so pip never replaces your ROCm PyTorch. On Linux it uses the triton that
comes with ROCm PyTorch; on Windows it pulls in triton-windows.
Uninstall upstream's CUDA sageattention first if it is there: both provide the module sageattention.
Windows, once: Triton also needs the ROCm SDK devel package that matches your ROCm wheels (HIP headers). It comes from AMD's repo.radeon.com, so pip cannot install it as a dependency. When it is missing, the first SageAttention call fails with the exact command to run, e.g. for ComfyUI 0.45's portable (ROCm 7.2.1):
python_embeded\python.exe -m pip install https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/rocm_sdk_devel-7.2.1-py3-none-win_amd64.whl
Python's include / libs folders, which the portable's embedded Python leaves out and Triton also needs, are
added automatically on first import (a one-time download from the triton-windows project).
From source
git clone -b rocm https://github.com/TwelveNights/SageAttention-rocm.git
pip install --no-deps --no-build-isolation ./SageAttention-rocm
setup.py detects the ROCm build of PyTorch and skips the CUDA extensions; with --no-build-isolation missing
setuptools, drop the flag and set SAGEATTN_ROCM=1 (the isolated build cannot import torch to detect ROCm).
SAGEATTN_ROCM=1 python -m build --wheel builds the PyPI wheel.
On RDNA3 the native kernel is compiled on first use with hipRTC (hiprtc*.dll / libhiprtc.so from the ROCm
runtime that PyTorch loaded -- no Visual Studio, no HIP SDK) in a few seconds and cached (see
Configuration). Without hipRTC it falls back to the Triton kernel.
Check the install
python -c "import torch; from sageattention import sageattn; q = torch.randn(1, 24, 4608, 128, device='cuda', dtype=torch.float16); o = sageattn(q, q, q); r = torch.nn.functional.scaled_dot_product_attention(q, q, q); print('cos', torch.nn.functional.cosine_similarity(o.flatten().float(), r.flatten().float(), dim=0).item())"
prints a cosine close to 1 (the first call on RDNA3 includes the native kernel's one-time compile). For the full test suite and benchmarks see Validation and benchmark scripts.
ComfyUI
ComfyUI already supports SageAttention through sageattention.sageattn; this port just has to be importable
from ComfyUI's Python.
-
Install it into ComfyUI's Python environment as above (portable:
python_embeded\python.exe -m pip ...; a venv / conda install: that environment'spip). If you set up ComfyUI yourself, install ComfyUI's own requirements without letting them replace ROCm PyTorch -- some of them (torchsde,kornia,spandrel) depend ontorchand pip may otherwise pull a CUDA build:python -c "import torch, torchvision; print(f'torch=={torch.__version__}\ntorchvision=={torchvision.__version__}')" > constraints.txt pip install -r ComfyUI/requirements.txt -c constraints.txt
-
Launch ComfyUI with the flag:
python main.py --use-sage-attention # Windows portable: add --use-sage-attention to the python_embeded\python.exe ... ComfyUI\main.py line # of the .bat launcher you use
-
Confirm it is active. The startup log must say
Using sage attention(instead of
Using pytorch attention). Ifsageattentioncannot be imported, ComfyUI logsTo use the --use-sage-attention feature, the sageattention package must be installed first.and keeps PyTorch attention. If a call into SageAttention fails at runtime, ComfyUI logsError running sage attention: ..., using pytorch attention instead.-- so check the log after the first generation too.example/rocm/comfyui_check.pychecks the patched function directly: it imports ComfyUI with the flag, asserts thatoptimized_attentionisattention_sage, and compares it withattention_pytorchon FLUX / Wan shapes:python example/rocm/comfyui_check.py --comfy /path/to/ComfyUI # sage enabled: True available: True supports mask: False sage3 available: False # optimized_attention -> attention_sage # B1 N4608 H24 D128 attention_sage cos 0.999781 ...
There is nothing to change in workflows: every model that goes through ComfyUI's optimized_attention
(Wan, HunyuanVideo, LTX-Video, FLUX, SD3, Qwen-Image, ...) uses SageAttention for its unmasked attention. Calls
with an attention mask (mostly text encoders) stay on PyTorch attention, as with upstream SageAttention on
NVIDIA. ComfyUI passes smooth_k=False, so its accuracy is slightly below calling sageattn() directly
(cos 0.9998 instead of 0.9999 in the check above); images are not visibly affected.
Example workflow. example/rocm/comfyui/wan2.1_t2v_1.3B.json
(API format: load it in the ComfyUI web UI or POST it to /prompt) is the stock Wan2.1 text-to-video graph
with ComfyUI's repackaged files from
Comfy-Org/Wan_2.1_ComfyUI_repackaged:
split_files/diffusion_models/wan2.1_t2v_1.3B_fp16.safetensors -> models/diffusion_models/,
split_files/text_encoders/umt5_xxl_fp8_e4m3fn_scaled.safetensors -> models/text_encoders/,
split_files/vae/wan_2.1_vae.safetensors -> models/vae/.
Measured with example/rocm/comfyui_ab.py, which starts ComfyUI once per
attention mode, runs the workflow three times with the same seeds in every mode and compares the frames:
python example/rocm/comfyui_ab.py --comfy /path/to/ComfyUI --workflow example/rocm/comfyui/wan2.1_t2v_1.3B.json \
--modes default,sage,kitchen --runs 3 --out runs/comfy_ab
| GPU (OS) | attention | sampler s/step | speedup | time per video (incl. VAE decode) | peak VRAM | PSNR vs default, runs 1-3 |
|---|---|---|---|---|---|---|
| RX 7800 XT (Windows 11) | default (Using pytorch attention) |
7.33 | 1.00x | 195.6 s | 13.1 GiB | reference |
| RX 7800 XT (Windows 11) | default + TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 |
7.42 | 0.99x | 208.7 s | 13.1 GiB | same frames as default |
| RX 7800 XT (Windows 11) | --use-sage-attention |
3.40 | 2.15x | 122.5 s | 13.1 GiB | 29.3 / 28.8 / 26.9 dB |
| RX 7800 XT (Windows 11) | ComfyUI's INT8 "comfy kitchen attention" (Model Attention Backend node) | 3.55 | 2.06x | 123.9 s | 13.1 GiB | 23.9 / 23.5 / 20.8 dB |
| W7900 (Ubuntu 24.04) | default (Using pytorch attention) |
5.16 | 1.00x | 105.2 s | 16.3 GiB | reference |
| W7900 (Ubuntu 24.04) | --use-sage-attention |
2.65 | 1.95x | 57.6 s | 16.3 GiB | 26.4 / 31.5 / 25.7 dB |
| W7900 (Ubuntu 24.04) | ComfyUI's INT8 "comfy kitchen attention" | 2.75 | 1.88x | 59.4 s | 16.2 GiB | 23.7 / 22.5 / 20.8 dB |
ComfyUI @ b1a247c, Wan2.1 T2V 1.3B fp16, 832x480, 33 frames, 20 steps, uni_pc, CFG 6, seeds 42-44; s/step is the median over runs 2-3 (run 1 includes loading and kernel compilation); "time per video" is ComfyUI's "Prompt executed in" (prompt encoding is cached after run 1, the VAE decode -- about 50 s on the 7800 XT -- is included); peak VRAM is device memory in use, polled. RX 7800 XT: torch 2.9.1 + ROCm 7.2.1 wheels, triton-windows 3.6.0.post26; W7900: torch 2.10.0+rocm7.0, Triton 3.6. Same seeds give the same scene, composition and motion in all modes; SageAttention's 26-31 dB to the default output is in the same range as the spread between exact attention implementations in the diffusers comparison below (27.6-32.6 dB), while ComfyUI's own INT8 attention, about as fast here, lands 4-6 dB further away. The default is PyTorch SDPA through ComfyUI's backend priority list (flash first); enabling PyTorch's experimental AOTriton kernels on top changes nothing in ComfyUI (row 2, separate session). Against a well-tuned FP16 flash attention (nunchaku-rocm's Triton kernel) the gain is smaller: 1.2-1.25x end to end (diffusers results below). A repeat session on the 7800 XT gave 3.49 and 3.76 s/step for SageAttention, plus one run at 11.3 s/step that was disturbed by other activity on the machine (not included above). On MI300X (Linux, torch 2.10.0+rocm7.0) the same workflow runs at 0.519 -> 0.391 s/step (1.33x; 12.9 -> 10.6 s per video), PSNR vs default 28.2 / 31.3 / 26.5 dB; ComfyUI's INT8 attention is not available there (the node falls back to PyTorch).
With nunchaku-rocm's nodes. nunchaku-rocm's
Nunchaku DiT Loader (ROCm) models (4-bit FLUX.1 / FLUX.2 [klein]) compute attention with nunchaku's own
fused Triton kernels, not through ComfyUI's optimized_attention, so --use-sage-attention does not change
them; it is safe to use the flag in the same ComfyUI (it then applies to every other model in your
workflows, e.g. a Wan video model next to a Nunchaku image model).
SageAttention3 (--use-sage-attention3) needs NVIDIA Blackwell FP4 hardware. Do not install
sageattention3_blackwell on AMD: it would only fall back to this INT8 kernel.
Python and diffusers
from sageattention import sageattn
# q, k, v: (batch, heads, seq, head_dim) fp16/bf16 on the AMD GPU ("HND"), or tensor_layout="NHD"
out = sageattn(q, k, v, tensor_layout="HND", is_causal=False)
All of upstream's entry points keep their signatures and dispatch to the AMD backend on a ROCm PyTorch:
sageattn, sageattn_qk_int8_pv_fp16_triton (also takes attn_mask), sageattn_qk_int8_pv_fp16_cuda,
sageattn_qk_int8_pv_fp8_cuda, sageattn_qk_int8_pv_fp8_cuda_sm90, sageattn_varlen. The *_cuda names are
historical; on AMD they select Q-scale granularity (per_warp = 32 rows, per_thread = 16) and FP16 vs FP8
P·V. Supported: NHD/HND with arbitrary strides, fp16/bf16, head dims up to 256 (33-128 on the native gfx11
kernel), causal, GQA/MQA, masks (Triton kernel), return_lse, varlen, qo_len != kv_len, torch.compile
(the Triton path is traced; the native kernel is a graph break). See ROCM_NOTES.md for
the kernel-level API and design notes.
diffusers (recent versions; checked with 0.40 and 0.41) has a SageAttention backend that calls sageattention.sageattn:
pipe = WanPipeline.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers", torch_dtype=torch.bfloat16).to("cuda")
pipe.transformer.set_attention_backend("sage")
(or with attention_backend("sage"): from diffusers.models.attention_dispatch for a block of code).
Checked on the W7900 and RX 7800 XT: the backend reaches sageattention.rocm, and the output matches the
default backend (cos 0.9999). example/rocm/wan_e2e.py runs Wan2.1 end to end with each attention
implementation (--attn sage-diffusers uses this backend) and compares the videos.
Results
All numbers below are reproducible with the scripts in this repository (commands in each section).
Kernel speed
TFLOP/s = 4·B·H·N²·D / time (non-causal), bf16 inputs, HND, "sage" includes quantization; best of 3
(bench/rocm/bench_attn.py). Baselines: the FP16 Triton flash attention from nunchaku-rocm (bench/rocm/nunchaku_triton_attn.py, vendored for
benchmarking), aiter's CK/assembly FlashAttention on MI300X, and PyTorch SDPA.
RX 7800 XT (gfx1101, Windows 11, ROCm 7.2.1, torch 2.9.1, triton-windows 3.6; "native" = the hipRTC kernel,
which sageattn() uses on gfx11):
| shape (B, H, N, D) | sage native | sage Triton | Triton FA fp16 (nunchaku) | torch SDPA |
|---|---|---|---|---|
| FLUX 1024² (1, 24, 4608, 128) | 4.93 ms (53) | 5.91 ms (44) | 6.55 ms (40) | 59.2 ms (4) |
| SD3-like (2, 24, 4096, 64) | 4.23 ms (49) | 5.09 ms (41) | 5.31 ms (39) | |
| FLUX 2048² (1, 24, 16896, 128) | 63.1 ms (56) | 75.0 ms (47) | 81.6 ms (43) | |
| Wan 1.3B 480p 41f (1, 12, 16380, 128) | 30.0 ms (55) | 36.6 ms (45) | 44.4 ms (37) | |
| Wan 1.3B 480p 81f (1, 12, 32760, 128) | 118.7 ms (56) | 144.6 ms (46) | 166.8 ms (40) | |
| same, NHD layout | 118.9 ms (55) | 146.6 ms (45) | ||
| (1, 24, 16384, 128) | 59.8 ms (55) | 70.5 ms (47) | 76.7 ms (43) | |
| causal (1, 24, 4608, 128) | 2.76 ms (47) | 3.29 ms (40) | ||
| causal (1, 12, 16380, 128) | 16.3 ms (50) | 18.7 ms (44) |
(TFLOP/s in parentheses.) The SDPA column is PyTorch's default on this setup; with
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 (enables its AOTriton flash kernels, marked experimental on RDNA3
in this PyTorch build) SDPA takes 17.0 ms (15 TFLOP/s) at FLUX 1024² instead of 55-59 ms. On the W7900 (Linux,
torch 2.10) the variable makes no difference. RDNA3's WMMA runs INT8 at the same rate as FP16 and has no FP8, so INT8 Q·K^T
buys no matrix throughput there; the speedup comes from the hand-written kernel, which reaches ~75% of the
7800 XT's 74.6 TFLOP/s FP16 peak (boost clock) at video shapes and ~71% at FLUX 1024².
Radeon PRO W7900 (gfx1100, Linux, ROCm 7.2.2 driver, torch 2.10 ROCm 7.0): native 66-87 TFLOP/s, Triton sage 59-70, Triton FA 53-64, torch SDPA 22-23 on the same shapes (e.g. Wan 1.3B 81f: 84-90 ms native, 98-107 ms Triton sage, 114-123 ms Triton FA, 292 ms SDPA; two sessions).
MI300X (gfx942, ROCm 7.2, torch 2.10, Triton 3.6; aiter 0.1.7 flash_attn_func; shared VM, ±10%):
| shape (B, H, N, D) | sage FP16 P·V | sage FP8 P·V | aiter bf16 | aiter FP8 (per-tensor) | Triton FA fp16 (nunchaku) | torch SDPA |
|---|---|---|---|---|---|---|
| FLUX 1024² (1, 24, 4608, 128) | 0.57 ms (459) | 0.51 ms (516) | 0.58 ms (451) | 0.42 ms (621) | 0.84 ms (311) | 0.85 ms (307) |
| SD3-like (2, 24, 4096, 64) | 0.57 ms (361) | 0.55 ms (375) | 0.66 ms (313) | -- | 0.76 ms (271) | 0.72 ms (288) |
| FLUX 2048² (1, 24, 16896, 128) | 6.67 ms (526) | 5.36 ms (654) | 7.30 ms (481) | 5.34 ms (658) | 10.85 ms (323) | 11.38 ms (308) |
| Wan 1.3B 480p (1, 12, 32760, 128) | 12.6 ms (524) | 10.4 ms (636) | 14.2 ms (465) | 10.7 ms (615) | 19.3 ms (342) | 22.3 ms (296) |
| Wan 14B 480p (1, 40, 32760, 128) | 38.7 ms (569) | 31.4 ms (700) | 40.7 ms (540) | 34.3 ms (640) | 58.2 ms (377) | 68.5 ms (321) |
| HunyuanVideo-like (1, 24, 45000, 128) | 43.7 ms (569) | 35.8 ms (694) | 45.8 ms (544) | 37.2 ms (668) | 65.6 ms (379) | 77.6 ms (321) |
| Wan 14B 720p (1, 40, 75600, 128) | 196 ms (598) | 161 ms (729) | 215 ms (545) | 179 ms (653) | 302 ms (388) | 349 ms (335) |
"aiter FP8" is a speed reference only (per-tensor FP8 Q/K/V cast outside the timed region, no scaling). Upstream's own Triton SageAttention, where it compiles on ROCm Triton 3.6 (head dim 64 only): 0.71 ms (289) on the SD3-like shape. Summary: the fastest of everything measured against the Triton FA and SDPA on all three GPUs; on MI300X the default FP16 P·V path is on par with aiter bf16 at FLUX 1024² and ~10% ahead at long video shapes, but slower than aiter's per-tensor FP8 kernel; sage FP8 P·V beats aiter FP8 only at
= ~32K tokens.
Accuracy
bench/rocm/accuracy.py: output vs FP32 attention, cosine similarity and relative L1 as in the SageAttention
papers. Synthetic inputs have a large per-channel K bias and a few 6x outlier Q/K channels, like real DiT
activations (B2 H8 N4096, bf16):
| head dim, mask | method | MI300X cos sim | MI300X rel-L1 | RX 7800 XT cos sim | RX 7800 XT rel-L1 |
|---|---|---|---|---|---|
| 64, full | sage, FP16 P·V | 0.999913 | 0.0097 | 0.999920 | 0.0095 |
| 64, full | sage, FP8 P·V (gfx942) | 0.999691 | 0.0215 | -- | -- |
| 64, full | upstream's own Triton SageAttention, run unmodified | 0.999899 | 0.0104 | 0.999899 | 0.0106 |
| 64, full | SDPA in bf16 (for scale) | 0.999999 | 0.0014 | 0.999999 | 0.0014 |
| 128, full | sage, FP16 P·V | 0.999898 | 0.0107 | 0.999901 | 0.0106 |
| 128, full | sage, FP8 P·V | 0.999652 | 0.0232 | -- | -- |
| 128, causal | sage, FP16 P·V | 0.999902 | 0.0103 | 0.999905 | 0.0101 |
| 128, causal | sage, FP8 P·V | 0.999643 | 0.0235 | -- | -- |
On the 7800 XT the native and Triton kernels give the same metrics to 6 digits (same quantization). The
default sageattn() granularity is per_warp (32-row Q scales), slightly finer than upstream's per-block
Triton path. Real activations -- self-attention Q/K/V captured from Wan2.1-T2V-1.3B (14040 tokens, 12 heads,
d=128) at 5 (step, block) points of a 20-step run (wan_e2e.py --dump-qkv, accuracy.py --qkv), MI300X:
| method | cos sim (min..max) | rel-L1 (min..max) |
|---|---|---|
| sage, FP16 P·V | 0.99993 .. 0.99998 | 0.0037 .. 0.0100 |
| sage, FP8 P·V | 0.99974 .. 0.99998 | 0.0062 .. 0.0201 |
| SDPA in bf16 | 0.999998 .. 0.999999 | 0.0011 .. 0.0017 |
FP8 P·V keeps P in e4m3 (3 mantissa bits), as upstream's FP8 kernels do; it has ~2x the FP16 path's error, so
sageattn() uses FP16 P·V by default on gfx942 and FP8 is opt-in.
End to end, diffusers (Wan2.1-T2V-1.3B)
example/rocm/wan_e2e.py: 480x832, 33 frames, 20 steps, CFG 5, same seed; prompts encoded once with UMT5-XXL;
initial noise from a CPU generator so runs on different GPUs start from identical latents. "PSNR vs
reference" compares the decoded 33-frame video with the MI300X bf16-SDPA run.
| GPU | attention | denoise (20 steps) | per step | speedup | PSNR vs reference | latent cos |
|---|---|---|---|---|---|---|
| RX 7800 XT | Triton FA fp16 (nunchaku) | 85.9 s | 4.20 s | 1.00x | 29.81 dB | 0.9975 |
| RX 7800 XT | SageAttention (native) | 70.9 s | 3.52 s | 1.21x | 26.80 dB | 0.9943 |
| W7900 | torch SDPA | 103.6 s | 5.19 s | 0.62x | 28.94 dB | 0.9964 |
| W7900 | Triton FA fp16 | 64.2 s | 3.22 s | 1.00x | 32.63 dB | 0.9989 |
| W7900 | SageAttention (native) | 55.3 s | 2.78 s | 1.16x | 29.83 dB | 0.9971 |
| MI300X | torch SDPA (bf16) | 12.1 s | 0.55 s | 1.00x | (reference) | 1 |
| MI300X | Triton FA fp16 | 11.6 s | 0.58 s | 1.04x | 30.25 dB | 0.9976 |
| MI300X | aiter FA (bf16) | 8.3 s | 0.41 s | 1.46x | 27.58 dB | 0.9943 |
| MI300X | SageAttention, FP16 P·V | 8.0 s | 0.40 s | 1.51x | 28.69 dB | 0.9964 |
| MI300X | SageAttention, FP8 P·V | 9.7 s | 0.48 s | 1.26x | 25.94 dB | 0.9925 |
Diffusion sampling amplifies any rounding difference, so the useful yardstick is the spread between exact attention implementations: they land 27.6-32.6 dB from the reference (SDPA itself on the W7900: 28.9 dB); SageAttention lands at 26.8-29.8 dB, and the frames show the same scene, composition and motion. At 81 frames (32760 tokens, 10 steps) the 7800 XT goes from 154.4 s to 123.7 s (1.25x). Peak VRAM while denoising is unchanged (3.9 GiB at 33 frames, 4.8 GiB at 81). The ComfyUI results are in the ComfyUI section.
3D generation (image to 3D model)
Same settings with PyTorch attention vs SageAttention, warm runs; the outputs match (same mesh and texture, rendered side by side).
| Model | GPU | PyTorch attention | SageAttention | speedup |
|---|---|---|---|---|
Pixal3D in ComfyUI (--use-sage-attention), full image -> textured GLB |
RX 7800 XT, Windows | 387 s | 215 s | 1.8x |
| Hunyuan3D-2.1 PBR texturing (Hunyuan3D-2.1-rocm) | W7900 | 64 s | 46 s | 1.4x |
| Hunyuan3D-2.1 shape, FlashVDM decoder | W7900 | 54 s | 42 s | 1.3x |
| TripoSG (TripoSG-rocm), 50 steps | W7900 | 37 s | 30 s | 1.2x |
TripoSG and the Hunyuan3D-2.1 painter in ComfyUI: ComfyUI-3D-rocm (uses SageAttention when installed).
In ComfyUI's TRELLIS.2 / Pixal3D the two high-resolution sampler stages (sparse attention over tens of thousands of tokens) go from 161 s to 68 s and from 88 s to 37 s.
Validation and benchmark scripts
| script | what it does |
|---|---|
scripts/rocm/validate.py [--bench] |
test suite + accuracy table (+ the kernel benchmarks above) on the local GPU; Linux and Windows |
tests/rocm/ (pytest tests/rocm) |
116 tests: NHD/HND, fp16/bf16, head dims 32-256, causal, GQA, masks, LSE, varlen, ragged lengths, native vs Triton, torch.compile (skipped on Windows, which has no host compiler for Inductor). 115 pass + 1 skip on the RX 7800 XT, 116 on the W7900; on MI300X the 70 gfx11-only tests skip |
bench/rocm/accuracy.py [--qkv file] |
accuracy table (synthetic or captured Q/K/V) |
bench/rocm/bench_attn.py --preset image|video|video_small |
kernel speed vs baselines (--methods, --shape B,H,N,D, --causal, --layout NHD) |
example/rocm/wan_e2e.py |
diffusers Wan2.1 end to end with each attention implementation, video comparison |
example/rocm/comfyui_check.py --comfy DIR |
ComfyUI's --use-sage-attention path at function level |
example/rocm/comfyui_ab.py --comfy DIR --workflow WF |
ComfyUI workflow A/B: default, default + TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1, --use-sage-attention, ComfyUI's INT8 "comfy kitchen" attention; timings, VRAM, PSNR |
Timings are only meaningful on an idle GPU; run validate.py once before validate.py --bench-only so the
Triton and hipRTC compile caches are warm.
Limitations
- SageAttention3 (FP4 microscaling,
sageattention3_blackwell/) needs NVIDIA Blackwell FP4 tensor cores and is not ported; on ROCmsageattn3_blackwell()falls back tosageattnwith a warning. - INT4 Q·K^T (SageAttention2's 4-bit variant; RDNA3 has 2x-rate INT4 WMMA) is not implemented: it costs accuracy and is not in upstream's current API.
- Causal attention at short lengths is not tuned on MI300X (B4 H32 N4096: 1.67 ms vs 1.27 ms for aiter bf16); only the non-causal path was tuned for DiTs. FP8 P·V (MI300X) is opt-in because of its accuracy.
- Masked attention runs on the Triton kernel (not the native gfx11 one); ComfyUI does not send masks to SageAttention at all (see ComfyUI).
- Tested on gfx942, gfx1100 and gfx1101 only. RDNA4 (gfx12), RDNA2, gfx90a and gfx950 run the generic Triton
path, untested. Windows support depends on
triton-windowsand AMD's ROCm wheels for Windows. - The first call per shape class compiles Triton kernels (and, on RDNA3, the native kernel once per GPU arch); later runs load them from the caches.
Configuration
| variable | effect |
|---|---|
SAGEATTN_NATIVE=0 / 1 |
RDNA3: disable the native hipRTC kernel (Triton only) / require it (raise instead of falling back) |
SAGEATTN_CACHE_DIR |
where compiled native kernels are cached (default ~/.cache/sageattention, %LOCALAPPDATA%\sageattention\cache on Windows) |
SAGEATTN_ROCM_PV=fp8 |
MI300X: FP8 P·V in sageattn() (default FP16) |
SAGEATTN_ROCM=1 |
install time: force the pure-Python ROCm install |
AMD_COMGR_CACHE_DIR |
ROCm's own hipRTC compiler cache (ROCm default: a per-user directory) |
TRITON_CACHE_DIR |
Triton's kernel cache |
Other SageAttention ports for AMD
This is not the only one. Upstream has open pull requests for AMD (thu-ml/SageAttention
#332 rocWMMA,
#368 native gfx12 backend,
#381 Triton path on ROCm), and upstream's SageAttention 1 on
PyPI (pip install sageattention, 1.0.6) is Triton-only and reported to run on ROCm. On RDNA4 (RX 9070, gfx1201)
prefer the ports with hand-tuned gfx1201 kernels, e.g.
IxMxAMAR/SageAttention-RDNA4 and the prebuilt Windows wheels
built from #368; this package runs its generic Triton path there. Other RDNA3 ports:
LuXuxue/sageattention-rdna3,
jnolck/SageAttention-rocm; earlier:
EmbeddedLLM/SageAttention-rocm. What this one adds: one
pure-Python package for RDNA3 and CDNA3 on Linux and Windows, a native RDNA3 kernel compiled at runtime (no
compiler toolchain), and the full SageAttention 2 API.
Credits and license
- SageAttention, SageAttention2/2++ and SageAttention3 are by Jintao Zhang, Jia Wei, Haofeng Huang, Pengle Zhang, Jun Zhu, Jianfei Chen and co-authors (Tsinghua University, thu-ml); please cite their papers (see README_upstream.md) if you use this work. This port re-implements their quantization scheme and kernels for AMD GPUs; the algorithms are theirs.
- The native-kernel infrastructure (
sageattention/rocm/native/hip.py,hiprtc.py,build.py,hip_prelude.h) is adapted from nunchaku-rocm, and the benchmark baselinebench/rocm/nunchaku_triton_attn.pyis its Triton flash attention, vendored unchanged except for one inlined helper (Apache-2.0). - License: Apache-2.0, same as upstream (LICENSE). The ROCm backend is new code under
sageattention/rocm/,bench/rocm/,example/rocm/,tests/rocm/andscripts/rocm/, plus small dispatch changes insageattention/core.py,setup.pyandsageattention3_blackwell/. - Not affiliated with or endorsed by the SageAttention authors or AMD.
Metadata
Release files for sageattention-rocm 2.2.0.post1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| sageattention_rocm-2.2.0.post1-py3-none-any.whl | Python 3 | none | any | Details for sageattention_rocm-2.2.0.post1-py3-none-any.whl |
Release files / sageattention_rocm-2.2.0.post1-py3-none-any.whl
| Download URL | sageattention_rocm-2.2.0.post1-py3-none-any.whl |
|---|---|
| Size | 78.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a931867fc402d1b195b86fc75e3db3932ade6b30017efe9f7b3d72a63fd1f62e
|
|
BLAKE2b-256 checksum How to use checksums |
3bc6d67b8786ebc85c648813216745fd8b2a036fc8369f25ee72b69abbce1058
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.3
|
Security for sageattention-rocm 2.2.0.post1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.