Skip to main content

sgemm-bi for PyTorch

Deterministic, batch-invariant CUDA GEMM for PyTorch training: bit-identical matmuls (forward / dW / dX) in float32, bfloat16, and float16, with an opt-in tensor-core tier that is faster than cuBLAS PEDANTIC on transformer-class shapes.

Built on the sgemm-bi Rust engine. No libtorch linkage — tensors cross as raw device pointers, so one wheel works with any PyTorch build. Kernels compile once at engine creation via NVRTC; the runtime needs only the NVIDIA driver.

Requirements: Linux x86_64, NVIDIA Ampere or newer (sm_80+), PyTorch with CUDA. (No macOS/Windows — there is no CUDA there; the package refuses to import with a clear message.)

Install

pip install maturin
cd python && maturin build --release
pip install target/wheels/sgemm_bi-*.whl

Use

import torch, sgemm_bi

# Drop-in layer (weight stored [in, out] — GEMM-natural; convert
# existing layers with Linear.from_torch):
layer = sgemm_bi.Linear(768, 3072, device="cuda", dtype=torch.bfloat16,
                        tensor_cores=True)
y = layer(x)
y.sum().backward()   # deterministic dW (f32-accumulated) and dX

# Functional form:
y = sgemm_bi.deterministic_linear(x, weight, bias, tensor_cores=True)

# Low-level engine (raw pointers, explicit stream):
eng = sgemm_bi.Engine(0)
eng.forward(y.data_ptr(), x.data_ptr(), w.data_ptr(), None,
            m, k, n, "bfloat16",
            torch.cuda.current_stream().cuda_stream, True)

Determinism contracts

tier guarantee
f32 / typed scalar bit-identical across runs; bf16/f16 ≡ "upcast → f32 kernel → RNE downcast"; full shape coverage
tensor cores (tensor_cores=True) own deterministic contract (mma.sync, f32 accumulate); bit-identical across runs; forward strictly batch-invariant across all M; falls back to the scalar tier when an output dim is < 64 (shape-only dispatch — still deterministic; two bit-identical tile families, 128×128 and 64×64, cover everything above)

Bias gradient is a plain f32 column sum done in torch (deterministic run-to-run for a fixed shape; not part of the engine's batch-invariance contract).

Examples

examples/train_deterministic.py trains twice from one seed and asserts bit-identical weights, then checks batch invariance. Typed stubs ship in the wheel (py.typed) — IDEs autocomplete and document the native Engine class.

Tests

pip install pytest && pytest tests/ -v   # needs a CUDA GPU

Metadata

Release files for sgemm-bi 0.1.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sgemm-bi 0.1.2
File Size Uploaded Provenance Actions
sgemm_bi-0.1.2.tar.gz 121.0 kB With provenance Details for sgemm_bi-0.1.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for sgemm-bi 0.1.2
File Interpreter ABI Platform Actions
sgemm_bi-0.1.2-cp39-abi3-manylinux_2_34_x86_64.whl CPython 3.9 abi3 Linux glibc 2.34+ x86-64 Details for sgemm_bi-0.1.2-cp39-abi3-manylinux_2_34_x86_64.whl

Total release size: 590.0 kB

Release files / sgemm_bi-0.1.2.tar.gz

Download URL sgemm_bi-0.1.2.tar.gz
Size 121.0 kB
Tags Source
SHA-256 checksum
How to use checksums
31414a9594dba0ccbd4558d6a871b8c4f122eda6744ec15919bc3f4a38d80d84
BLAKE2b-256 checksum
How to use checksums
118203979af9909c3647fef603f5ab6514c65fd588b463d6bd2ed65fce1d0561
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 2, 2026.

Transparency log

Release files / sgemm_bi-0.1.2-cp39-abi3-manylinux_2_34_x86_64.whl

Download URL sgemm_bi-0.1.2-cp39-abi3-manylinux_2_34_x86_64.whl
Size 469.1 kB
Tags CPython 3.9 Linux glibc 2.34+ x86-64 abi3
SHA-256 checksum
How to use checksums
dbee301eb11c0df70daa2f7f3d2c09697a91375bbc85ad0dd291f282a9721265
BLAKE2b-256 checksum
How to use checksums
ef88a50f51dbb1d943aa85d5a5344ef21fba76cd6152010e17dde6aed0e21c10
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 2, 2026.

Transparency log

Security for sgemm-bi 0.1.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for sgemm_bi-0.1.2.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 2, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page