h3-sparse-attn
Block-sparse self-attention for the MiniMax H3 video DiT. Each query block attends to its top-k key blocks of 128 tokens; the skipped blocks are folded back in as one summary row each. Training-free, so the released H3 checkpoint runs unchanged.
Requires sm_120 (RTX 5090, RTX PRO 6000 Blackwell), bf16, head_dim 128, forward only. Anything else falls back to the attention H3 already ships.
This is an approximation: same quality, but not the same sample as full attention at a fixed seed. top_k sets how
far it drifts.
Install
pip install h3-sparse-attn
Use
import h3_sparse_attn
h3_sparse_attn.patch(model, top_k=64)
model is a MiniMaxH3Transformer3DModel or a pipeline holding one. top_k counts 128-token key blocks.
Measured
RTX PRO 6000 Blackwell Server Edition, bf16, 56 heads, head_dim 128, top_k=64. Sequence lengths are what H3 packs
at 24 fps. "Step" is one denoising step of the 50-layer DiT.
| Video | Sequence | Attention | Speedup | Step | Speedup |
|---|---|---|---|---|---|
| 768p 5s | 37,740 | 112.4 → 30.7 ms | 3.7x | 10.67 → 6.47 s | 1.6x |
| 768p 10s | 73,416 | 426.6 → 61.8 ms | 6.9x | 31.26 → 12.70 s | 2.5x |
| 768p 15s | 103,996 | 862.7 → 91.0 ms | 9.5x | 57.12 → 18.11 s | 3.2x |
| 2K 5s | 133,644 | 1435.6 → 121.8 ms | 11.8x | 89.22 → 23.48 s | 3.8x |
License
MIT
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file h3_sparse_attn-0.1.0.tar.gz.
File metadata
- Download URL: h3_sparse_attn-0.1.0.tar.gz
- Upload date:
- Size: 13.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7843a8d49cc551db3c741dc9bce889d32adf82ab53590ce84957072c970aa46b
|
|
| MD5 |
a752c8409e6c151d32725069572b3b28
|
|
| BLAKE2b-256 |
b45641eadd0aed4c2d3e4d1a6038e85f7bdb6805338c10dbf16149abb59e2182
|
Provenance
The following attestation bundles were made for h3_sparse_attn-0.1.0.tar.gz:
Publisher:
publish.yml on Occipital-Labs/h3-sparse-attn
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
h3_sparse_attn-0.1.0.tar.gz -
Subject digest:
7843a8d49cc551db3c741dc9bce889d32adf82ab53590ce84957072c970aa46b - Sigstore transparency entry: 2531650472
- Sigstore integration time:
-
Permalink:
Occipital-Labs/h3-sparse-attn@4ad4541c54c4d68657ec0d2bafa12c09f6d8ff62 -
Branch / Tag:
refs/tags/v0.1.0 - Owner: https://github.com/Occipital-Labs
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@4ad4541c54c4d68657ec0d2bafa12c09f6d8ff62 -
Trigger Event:
push
-
Statement type:
File details
Details for the file h3_sparse_attn-0.1.0-py3-none-any.whl.
File metadata
- Download URL: h3_sparse_attn-0.1.0-py3-none-any.whl
- Upload date:
- Size: 13.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
fd27b0d68e77d4e3bd0b6798482e311a284fa598e1850289d2dd2dcddf1057d6
|
|
| MD5 |
e58aa0c287bdd5387e177a0f87f1ba12
|
|
| BLAKE2b-256 |
4847f3215d68e07f1d1a25fa21658df46bc6a70f3228a8055991771606631561
|
Provenance
The following attestation bundles were made for h3_sparse_attn-0.1.0-py3-none-any.whl:
Publisher:
publish.yml on Occipital-Labs/h3-sparse-attn
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
h3_sparse_attn-0.1.0-py3-none-any.whl -
Subject digest:
fd27b0d68e77d4e3bd0b6798482e311a284fa598e1850289d2dd2dcddf1057d6 - Sigstore transparency entry: 2531651284
- Sigstore integration time:
-
Permalink:
Occipital-Labs/h3-sparse-attn@4ad4541c54c4d68657ec0d2bafa12c09f6d8ff62 -
Branch / Tag:
refs/tags/v0.1.0 - Owner: https://github.com/Occipital-Labs
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@4ad4541c54c4d68657ec0d2bafa12c09f6d8ff62 -
Trigger Event:
push
-
Statement type:
