grouped_mm_bf16 (Triton)
Install
-
Install PyTorch for your system (CUDA / ROCm / CPU): https://pytorch.org/get-started/locally/
-
Install this package (and Triton):
pip install grouped-mm-bf16[triton]
Usage
import torch
from grouped_mm_bf16 import grouped_mm
dev = "cuda"
# 3D x 3D (batched matmul): [G,M,K] @ [G,K,N] -> [G,M,N]
a = torch.randn((4, 32, 64), device=dev, dtype=torch.bfloat16)
b = torch.randn((4, 64, 48), device=dev, dtype=torch.bfloat16)
out = grouped_mm(a, b)
# 2D x 3D with offsets: [M_total,K] @ [G,K,N] -> [M_total,N]
sizes = torch.tensor([0, 7, 1, 13, 0, 11], device=dev, dtype=torch.int32)
offs = sizes.cumsum(0) # group end indices along M_total
a = torch.randn((int(offs[-1].item()), 64), device=dev, dtype=torch.bfloat16)
b = torch.randn((sizes.numel(), 64, 48), device=dev, dtype=torch.bfloat16)
out = grouped_mm(a, b, offs=offs)
About
Universal (NVIDIA CUDA + AMD HIP) Triton implementation of torch.nn.functional.grouped_mm semantics for BF16.
This repo provides grouped_mm_bf16.grouped_mm(...), mirroring the private operator signature:
grouped_mm(mat_a, mat_b, offs=None, bias=None, out_dtype=None) -> Tensor
Notes:
- GPU-only: raises if any inputs are on CPU; never falls back to CPU.
- Matches PyTorch grouped_mm shape semantics (2D/3D +
offs) and dtype rules. biasis rejected, matching current PyTorch behavior (Bias not supported yet).- Like PyTorch’s current CUDA implementation, inputs must have “valid” (row/column-major) strides and 16-byte alignment; contiguous matrices with odd inner dimensions may need padding.
Metadata
Release files for grouped-mm-bf16 0.1.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| grouped_mm_bf16-0.1.7-py3-none-any.whl | Python 3 | none | any | Details for grouped_mm_bf16-0.1.7-py3-none-any.whl |
Release files / grouped_mm_bf16-0.1.7-py3-none-any.whl
| Download URL | grouped_mm_bf16-0.1.7-py3-none-any.whl |
|---|---|
| Size | 16.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
98952cdaba973dc0f31f171ef9b8fd8975de697646cb763c3d8983c6b20c3074
|
|
BLAKE2b-256 checksum How to use checksums |
bffc3d820981687a1ccaa98f1c6776457d1f38cc8e2c7bd556228e350fdecac9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.10
|
Security for grouped-mm-bf16 0.1.7
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.