torch-einshard
torch-einshard expresses local and distributed PyTorch tensor operations with
einsum-like axis and sharding notation.
Installation
python -m pip install torch-einshard
The main entry point is torch_einshard.einshard:
import torch_einshard as es
z = es.einshard("a b, b c -> a c", x, y)
Axis names work like torch.einsum subscripts. Sharded logical axes can also
name a PyTorch DeviceMesh dimension with /:
a/sp b/dp, b/dp c -> a/sp c
This means axis a is sharded over mesh dimension sp, axis b is sharded
over dp, and the contraction over b/dp produces partial local results that
are all-reduced over dp.
Tensor-level partial values use //:
b n h // tp -> b n h
A partial tensor has the full logical shape locally, but each rank holds only one contribution to the value. Converting it back to a non-partial tensor sum-reduces over the named mesh dimension.
Quick Examples
Local contraction:
z = es.einshard("a b k c, b c l d -> k l d a", x, y)
Distributed split:
from torch.distributed.device_mesh import init_device_mesh
mesh = init_device_mesh("cpu", (2, 4), mesh_dim_names=("dp", "sp"))
z = es.einshard("a b -> a/sp b", x, mesh=mesh, shapes={"sp": a_shapes})
Tensor-parallel contraction:
z = es.einshard("b n c, h/tp c -> b n h/tp", x, weight_shard, mesh=mesh)
Named-axis FFT:
z = es.einfft("b x y c -> b kx ky c", x, axes={"x": "kx", "y": "ky"})
Halo exchange and local-window construction:
patches = es.einwindow(
"b h/sp_h w/sp_w c -> b h/sp_h w/sp_w kh kw c",
x,
{"h": "kh", "w": "kw"},
{"h": 1, "w": 1},
mesh=mesh,
shapes={"sp_h": h_shapes, "sp_w": w_shapes},
)
Documentation
The full documentation lives under
docs/:
- Getting started covers installation, quickstart, and test commands.
- Notation covers axis, sharding, partial, factored-axis, family, and ellipsis notation.
- Meshes and shapes
covers
DeviceMesh, compound mesh groups, and uneven split metadata. - User guides cover local, distributed, FFT, halo/window/convolution, roll, parameter, and policy topics.
- Reference documentation covers the API and known limitations.
- Development documentation covers the roadmap, performance, and design notes.
Build the HTML docs with:
uv run sphinx-build -b html docs docs/_build/html
Running Tests
Local and single-process tests:
uv run pytest
Full distributed test suite:
./run_tests.sh
run_tests.sh uses torchrun --nproc-per-node 8.
Current Status
torch-einshard supports local einsum-style operations, selected distributed
split/gather/repartition patterns, tensor-parallel contractions, partial tensor
notation, named-axis FFTs, halo/window/convolution helpers, roll operations,
optimization-policy diagnostics, and parameter metadata helpers.
Known limitations are tracked in the limitations reference. Remaining feature and planning work is tracked in the roadmap.
Metadata
Release files for torch-einshard 1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| torch_einshard-1.0.tar.gz | 51.7 kB | Without provenance | Details for torch_einshard-1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| torch_einshard-1.0-py3-none-any.whl | Python 3 | none | any | Details for torch_einshard-1.0-py3-none-any.whl |
Total release size: 111.0 kB
Release files / torch_einshard-1.0.tar.gz
| Download URL | torch_einshard-1.0.tar.gz |
|---|---|
| Size | 51.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0754d85a7bbac0826af0d35510792cfaefcff128dde5c76f59c90e4c5c52cb87
|
|
BLAKE2b-256 checksum How to use checksums |
86136f748ef8417c756dc7624154241c562e7954be97b76767006fbad29e097c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.14.6
|
Release files / torch_einshard-1.0-py3-none-any.whl
| Download URL | torch_einshard-1.0-py3-none-any.whl |
|---|---|
| Size | 59.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
87392fc94273fa39ace1daa1334a96a2b2e5156266ab62235bcc7116661ff173
|
|
BLAKE2b-256 checksum How to use checksums |
e9676f8c69207369c53534d013a3c83f336944f5106602f2e08204a6063d9da5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.14.6
|
Security for torch-einshard 1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.