TensorTorrent

Heterogeneous execution planning for PyTorch on a single machine.
Profile → plan → simulate → compile → run across CPUs, GPUs, memory, and storage.

CI PyPI PyPI downloads Python 3.10–3.13 Rust native planner Linux Apache-2.0 GitHub stars

Install · Quick start · When to use · Docs · Contributing

TensorTorrent profiles a host, searches placements in a native Rust planner, simulates the strongest schedules, then compiles and runs the winner — across unequal CPUs, accelerators, memory, and storage.

[!NOTE] Alpha. CPU and virtual backends are covered by CI. Validate accelerators on the target host with tensortorrent validate-hardware.

TensorTorrent compilation and execution pipeline

Install

pip install torch
pip install tensortorrent

Linux · Python 3.10–3.13 · PyTorch 2.4+. Source builds and CUDA/ROCm/XPU notes: Installation.

Quick start

import torch
import torch.nn as nn
import tensortorrent as tt

model = nn.Sequential(
    nn.Linear(256, 1024),
    nn.GELU(),
    nn.Linear(1024, 256),
).eval()

x = torch.randn(32, 256)
compiled = tt.compile(model, example_inputs=(x,))

y = compiled(x)
torch.testing.assert_close(y, model(x), check_device=False)
print(compiled.explain())

Save/reload, objectives, and multi-module graphs: Quickstart.

When to use

Good fit: model does not fit one GPU · unequal devices · transfer cost matters · RAM/VRAM budgets · parameter streaming or activation spill · reproducible plans instead of hand-written .to(device) maps.

Not for: multi-node clusters · exhaustive placement search · “use every detected GPU” · replacing PyTorch kernels · treating discovery as production validation.

Full boundary: Product scope.

Benchmark snapshot (MEASURED)

Host: RTX 3070 Ti Laptop 8 GiB, 61 GiB RAM, PyTorch 2.13, package 0.3.0. Measured on clean commit 308dd8fdb58d (git_dirty=false). Frozen JSON: benchmarks/published/2026-08-09/. Details: Benchmarks.

Workload Eager / baseline TensorTorrent Peak VRAM Notes
DeepMLP 1.5× VRAM (12.35 GB) GPU OOM (probe); tested Accelerate 940 ms; CPU 756 ms 1634 ms 0.61 GB Capacity / GPU compute on this PCIe laptop — not a latency win vs CPU/Accelerate
Qwen3-8B bf16 logits forward seq16 (16.38 GB) infeasible by param footprint; tested Accelerate OOM'd; CPU 5741 ms 2736 ms 1.33 GB Not autoregressive generation; fixed-shape exportable forward only
MLP 512×8 (fits) eager 0.23 ms 1.12 ms 17 MB TT slower when model fits
MLP 2048×8 (fits) eager 0.71 ms 1.22 ms 143 MB overhead shrinks on heavier forwards

2× GPU / ROCm / XPU / autoregressive generation: SUPPORTED BUT UNMEASURED on this machine.

Docs

Start at the documentation index.
Getting started Install · Quickstart
Architecture Overview · Planner · Runtime
Ops Large models · Deployment · FAQ 
tensortorrent doctor
tensortorrent validate-hardware --output artifacts/validation_report.json
make check   # from a source checkout

License

Apache License 2.0. See LICENSE.

TensorTorrent icon
Made for mixed machines — not just mixed kernels.

Project links

Data verified by PyPI on Aug 9, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.3.4)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ajvendetta from gravatar.com ajvendetta

Credits

Author: TensorTorrent Contributors

GitHub Statistics

Data verified by PyPI on Aug 9, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python >=3.10

Provides Extra

bench dev

Tags

pytorch compiler inference heterogeneous multi-gpu numa streaming

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tensortorrent-0.3.0.tar.gz (391.9 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

tensortorrent-0.3.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

tensortorrent-0.3.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.5 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ ARM64

tensortorrent-0.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

tensortorrent-0.3.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.5 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ ARM64

tensortorrent-0.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

tensortorrent-0.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

File details

Details for the file tensortorrent-0.3.0.tar.gz.

File metadata

  • Download URL: tensortorrent-0.3.0.tar.gz
  • Upload date:
  • Size: 391.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tensortorrent-0.3.0.tar.gz
Algorithm Hash digest
SHA256 76e3df0086654a2e69a64ba3234025022e1aafc6fd53486a665586ddf6307643
MD5 8256b56986d2231ecfdee078ea17fc0d
BLAKE2b-256 90652b656a7fdc9e5f98b3a4d9c8d81b608240261b584091a75cadbb8f631cfe

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.0.tar.gz:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file tensortorrent-0.3.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tensortorrent-0.3.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 275a3f4bdfec1bbc48d128b13079d18c6fd35b2f826cec057e8a5d953db3bf35
MD5 6adb61cc5c58dc57054365b01e1b7bb0
BLAKE2b-256 95d7dc1dd9969a9689742e29d6115a334f762cb10630d883c6f5dafadad52601

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file tensortorrent-0.3.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tensortorrent-0.3.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 10526f4dd23eedc0bea4d746cb51bab6036eed164bafdf2e51b2726d34271b04
MD5 14cbaa8484ad40d69732798b19447fe4
BLAKE2b-256 76f5a4a2479c8423ac72f178558656a0d0d615a29f1bbcd4afd395867ad9e802

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file tensortorrent-0.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tensortorrent-0.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 3eaa489c0cc7204d644a992b1686bfdcaeea564a38d27fe1988a80de98ebcbf1
MD5 05465a7906015de5a2eb0538785db52a
BLAKE2b-256 1ab5d6f8835f5fcb87d65624e04a40bd9b755cee9647224a471a70835dfc8ddd

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file tensortorrent-0.3.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tensortorrent-0.3.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 3839b9796f09d7d1999b1e7b600a6d8150b8855d8edd6a3799e47505a0b2e666
MD5 77f35f0317460d0d13264247a04bc977
BLAKE2b-256 11fc8a55f625f6136be212ea0b72fd8f6d5a9d836a8d5c4810658d271e39942e

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file tensortorrent-0.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tensortorrent-0.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 083719d5bdd53e1ec0f733453a6baa3a2b777b9e083acb2455488a203713f8c6
MD5 e4f8472fa171b945272417f3deca74d2
BLAKE2b-256 1cd296183b7d14b38f8e2acd602cff040bb0eb0e0c5111dffb1a11286f3b63f2

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file tensortorrent-0.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tensortorrent-0.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 9861e97320a49bf2ea94ab360d4479758709fdf9f60056a2fb4e0ee97e6a9f5c
MD5 e1af5f7fb6a87fefbadb2dfcdadd11a5
BLAKE2b-256 2b527e1c55a6015dc0cd368faa1534c6f5126b77214c5893e91fbe97f2f3a310

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page