Skip to main content

grouped_mm_bf16 (Triton)

Install

  1. Install PyTorch for your system (CUDA / ROCm / CPU): https://pytorch.org/get-started/locally/

  2. Install this package (and Triton):

pip install grouped-mm-bf16[triton]

Usage

import torch
from grouped_mm_bf16 import grouped_mm

dev = "cuda"

# 3D x 3D (batched matmul): [G,M,K] @ [G,K,N] -> [G,M,N]
a = torch.randn((4, 32, 64), device=dev, dtype=torch.bfloat16)
b = torch.randn((4, 64, 48), device=dev, dtype=torch.bfloat16)
out = grouped_mm(a, b)

# 2D x 3D with offsets: [M_total,K] @ [G,K,N] -> [M_total,N]
sizes = torch.tensor([0, 7, 1, 13, 0, 11], device=dev, dtype=torch.int32)
offs = sizes.cumsum(0)  # group end indices along M_total
a = torch.randn((int(offs[-1].item()), 64), device=dev, dtype=torch.bfloat16)
b = torch.randn((sizes.numel(), 64, 48), device=dev, dtype=torch.bfloat16)
out = grouped_mm(a, b, offs=offs)

About

Universal (NVIDIA CUDA + AMD HIP) Triton implementation of torch.nn.functional.grouped_mm semantics for BF16.

This repo provides grouped_mm_bf16.grouped_mm(...), mirroring the private operator signature:

grouped_mm(mat_a, mat_b, offs=None, bias=None, out_dtype=None) -> Tensor

Notes:

  • GPU-only: raises if any inputs are on CPU; never falls back to CPU.
  • Matches PyTorch grouped_mm shape semantics (2D/3D + offs) and dtype rules.
  • bias is rejected, matching current PyTorch behavior (Bias not supported yet).
  • Like PyTorch’s current CUDA implementation, inputs must have “valid” (row/column-major) strides and 16-byte alignment; contiguous matrices with odd inner dimensions may need padding.

Metadata

Release files for grouped-mm-bf16 0.1.7

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for grouped-mm-bf16 0.1.7
File Interpreter ABI Platform Actions
grouped_mm_bf16-0.1.7-py3-none-any.whl Python 3 none any Details for grouped_mm_bf16-0.1.7-py3-none-any.whl

Release files / grouped_mm_bf16-0.1.7-py3-none-any.whl

Download URL grouped_mm_bf16-0.1.7-py3-none-any.whl
Size 16.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
98952cdaba973dc0f31f171ef9b8fd8975de697646cb763c3d8983c6b20c3074
BLAKE2b-256 checksum
How to use checksums
bffc3d820981687a1ccaa98f1c6776457d1f38cc8e2c7bd556228e350fdecac9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.10

Security for grouped-mm-bf16 0.1.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page