tensortorrent

TensorTorrent

Heterogeneous execution planning for PyTorch on a single machine.
Profile → plan → simulate → compile → run across CPUs, GPUs, memory, and storage.

Python 3.10–3.13

Install · Quick start · When to use · Docs · Contributing

TensorTorrent is a capacity-oriented heterogeneous runtime for PyTorch. It profiles a host, searches placements in a native Rust planner, simulates finalists, then compiles and runs the winner across unequal CPUs, accelerators, memory tiers, and storage.

Problem: when a model approaches or exceeds accelerator memory, hand-written .to(device) maps and ad-hoc offload become fragile. TensorTorrent can trade PCIe/host transfer bandwidth for execution capacity by streaming state through the accelerator.

[!NOTE] Alpha. CPU and virtual backends are covered by CI. Validate accelerators on the target host with tensortorrent validate-hardware.

Headline capacity result (MEASURED)

Qwen3-8B BF16 contains 16.38 GB of parameters. On an RTX 3070 Ti Laptop GPU with ~8 GiB VRAM, TensorTorrent executes a fixed-shape logits forward (seq_len=16) while keeping peak allocated GPU memory around 1.33 GB — by streaming / Transfer–Evict, not by fitting the full model in VRAM.

This is not autoregressive generation. Native PyTorch is generally faster when the model fits comfortably in one GPU. Raw evidence (commit fb503e5, git_dirty=false, package 0.3.1): benchmarks/published/2026-08-09/ · Benchmarks.

Workload	Eager / baseline	TensorTorrent	Peak VRAM	Notes
Qwen3-8B bf16 logits forward seq16 (16.38 GB)	infeasible by param footprint; tested Accelerate OOM'd; CPU 2119 ms	2522 ms	1.33 GB	cosine 0.9997, argmax 15/16; fixed-shape only
DeepMLP 1.5× VRAM (12.35 GB)	GPU OOM (probe); tested Accelerate 916 ms; CPU 734 ms	1580 ms	0.61 GB	capacity / GPU compute — not a latency win vs CPU/Accelerate here
MLP 512×8 (fits)	eager 0.23 ms	1.09 ms	17 MB	TT slower when model fits
MLP 2048×8 (fits)	eager 0.71 ms	1.24 ms	143 MB	overhead shrinks on heavier forwards

2× GPU / ROCm / XPU / autoregressive generation: SUPPORTED BUT UNMEASURED on this machine.

TensorTorrent compilation and execution pipeline

Install

pip install torch
pip install tensortorrent

Linux · Python 3.10–3.13 · PyTorch 2.4+. Source builds and CUDA/ROCm/XPU notes: Installation.

Quick start

import torch
import torch.nn as nn
import tensortorrent as tt

model = nn.Sequential(
    nn.Linear(256, 1024),
    nn.GELU(),
    nn.Linear(1024, 256),
).eval()

x = torch.randn(32, 256)
compiled = tt.compile(model, example_inputs=(x,))

y = compiled(x)
torch.testing.assert_close(y, model(x), check_device=False)
print(compiled.explain())

Save/reload, objectives, and multi-module graphs: Quickstart.

When to use

Good fit: model does not fit one GPU · unequal devices · transfer cost matters · RAM/VRAM budgets · parameter streaming or activation spill · reproducible plans instead of hand-written device maps.

Not for: multi-node clusters · exhaustive placement search · “use every detected GPU” · replacing PyTorch kernels · treating discovery as production validation.

Full boundary: Product scope.

Docs


Getting started	Install · Quickstart
Architecture	Overview · Planner · Runtime
Ops	Large models · Deployment · FAQ

tensortorrent doctor
tensortorrent validate-hardware --output artifacts/validation_report.json
make check   # from a source checkout

License

Apache License 2.0. See LICENSE.

TensorTorrent icon
_{TensorTorrent}

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tensortorrent-0.3.1.tar.gz (392.5 kB view details)

Uploaded Aug 9, 2026 Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB view details)

Uploaded Aug 9, 2026 CPython 3.13manylinux: glibc 2.17+ x86-64

tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.5 MB view details)

Uploaded Aug 9, 2026 CPython 3.13manylinux: glibc 2.17+ ARM64

tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB view details)

Uploaded Aug 9, 2026 CPython 3.12manylinux: glibc 2.17+ x86-64

tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.5 MB view details)

Uploaded Aug 9, 2026 CPython 3.12manylinux: glibc 2.17+ ARM64

tensortorrent-0.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB view details)

Uploaded Aug 9, 2026 CPython 3.11manylinux: glibc 2.17+ x86-64

tensortorrent-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB view details)

Uploaded Aug 9, 2026 CPython 3.10manylinux: glibc 2.17+ x86-64

File details

Details for the file tensortorrent-0.3.1.tar.gz.

File metadata

Download URL: tensortorrent-0.3.1.tar.gz
Upload date: Aug 9, 2026
Size: 392.5 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tensortorrent-0.3.1.tar.gz
Algorithm	Hash digest
SHA256	`868f6c77428c0c5aa2110bf8ea71989606cf9afb8d7297ec23a8cd01d5dbda97`
MD5	`418627ed726ed2c55cecc5e689a088bd`
BLAKE2b-256	`9a977b9623d3732ebf338d25ef0d8c17f23bfaa2d331df85b36358aa6f7d468a`

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.1.tar.gz:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: tensortorrent-0.3.1.tar.gz
- Subject digest: 868f6c77428c0c5aa2110bf8ea71989606cf9afb8d7297ec23a8cd01d5dbda97
- Sigstore transparency entry: 2397115825
- Sigstore integration time: Aug 9, 2026
Source repository:
- Permalink: alhussein-jamil/TensorTorrent@384b20ae0ec0536e6319965b5addd808086dbe1a
- Branch / Tag: refs/tags/v0.3.1
- Owner: https://github.com/alhussein-jamil
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@384b20ae0ec0536e6319965b5addd808086dbe1a
- Trigger Event: push

File details

Details for the file tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

Download URL: tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Upload date: Aug 9, 2026
Size: 1.5 MB
Tags: CPython 3.13, manylinux: glibc 2.17+ x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm	Hash digest
SHA256	`91d51502cd954d23d41a2bd4a5f66ad509578e50c606cd3bef5ae29049e1c1ca`
MD5	`a4588962ead3f95b5983b0574caae024`
BLAKE2b-256	`e4e3436ac2da9cd46c3517bcaa9ef7562aa7ff72a97d689ab9e2b94a6d8b7e80`

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Subject digest: 91d51502cd954d23d41a2bd4a5f66ad509578e50c606cd3bef5ae29049e1c1ca
- Sigstore transparency entry: 2397115976
- Sigstore integration time: Aug 9, 2026
Source repository:
- Permalink: alhussein-jamil/TensorTorrent@384b20ae0ec0536e6319965b5addd808086dbe1a
- Branch / Tag: refs/tags/v0.3.1
- Owner: https://github.com/alhussein-jamil
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@384b20ae0ec0536e6319965b5addd808086dbe1a
- Trigger Event: push

File details

Details for the file tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

Download URL: tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Upload date: Aug 9, 2026
Size: 1.5 MB
Tags: CPython 3.13, manylinux: glibc 2.17+ ARM64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm	Hash digest
SHA256	`58bef69708ba9ee6a9c0e0196008b13d24fcbebbb10b338186679d85f382b4f4`
MD5	`932fe73a41b358ca2b5d3edf87136769`
BLAKE2b-256	`7a5cd30c832a440c1dde4225cc5a8f2b4dd6817992d448b8aea89d3eec627cc2`

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: tensortorrent-0.3.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- Subject digest: 58bef69708ba9ee6a9c0e0196008b13d24fcbebbb10b338186679d85f382b4f4
- Sigstore transparency entry: 2397116308
- Sigstore integration time: Aug 9, 2026
Source repository:
- Permalink: alhussein-jamil/TensorTorrent@384b20ae0ec0536e6319965b5addd808086dbe1a
- Branch / Tag: refs/tags/v0.3.1
- Owner: https://github.com/alhussein-jamil
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@384b20ae0ec0536e6319965b5addd808086dbe1a
- Trigger Event: push

File details

Details for the file tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

Download URL: tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Upload date: Aug 9, 2026
Size: 1.5 MB
Tags: CPython 3.12, manylinux: glibc 2.17+ x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm	Hash digest
SHA256	`94261dcc5b5d4b84589875670107c2052653a40ad9edebd3ddc8f340cdc08bb7`
MD5	`064a9cf54790c52c7dc02b9bea238c7c`
BLAKE2b-256	`9d5bc22ef34f1b38e29e92e0c5d89be0090488611bc0e3907b2728e82e05ffff`

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Subject digest: 94261dcc5b5d4b84589875670107c2052653a40ad9edebd3ddc8f340cdc08bb7
- Sigstore transparency entry: 2397116200
- Sigstore integration time: Aug 9, 2026
Source repository:
- Permalink: alhussein-jamil/TensorTorrent@384b20ae0ec0536e6319965b5addd808086dbe1a
- Branch / Tag: refs/tags/v0.3.1
- Owner: https://github.com/alhussein-jamil
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@384b20ae0ec0536e6319965b5addd808086dbe1a
- Trigger Event: push

File details

Details for the file tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

Download URL: tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Upload date: Aug 9, 2026
Size: 1.5 MB
Tags: CPython 3.12, manylinux: glibc 2.17+ ARM64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm	Hash digest
SHA256	`eb06b9f885402652b874e9b8b2ee815b7574bf648a95490cb74cff0d23bd81c4`
MD5	`188b2e5ebbd5d200b832711a902ec72f`
BLAKE2b-256	`8c1fa255217f28a4b602056ab15ccd9076b22dff385a7f9fcfff2a27f033083f`

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: tensortorrent-0.3.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- Subject digest: eb06b9f885402652b874e9b8b2ee815b7574bf648a95490cb74cff0d23bd81c4
- Sigstore transparency entry: 2397115902
- Sigstore integration time: Aug 9, 2026
Source repository:
- Permalink: alhussein-jamil/TensorTorrent@384b20ae0ec0536e6319965b5addd808086dbe1a
- Branch / Tag: refs/tags/v0.3.1
- Owner: https://github.com/alhussein-jamil
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@384b20ae0ec0536e6319965b5addd808086dbe1a
- Trigger Event: push

File details

Details for the file tensortorrent-0.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

Download URL: tensortorrent-0.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Upload date: Aug 9, 2026
Size: 1.5 MB
Tags: CPython 3.11, manylinux: glibc 2.17+ x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tensortorrent-0.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm	Hash digest
SHA256	`f6b3a17d6d32983f860e417fba4b5edc284db38bfa7d2921dfee95a9ea7f9baa`
MD5	`45f1b18508eb4793e7e0e1e4ca228170`
BLAKE2b-256	`f8ce2363f3319e6a1bd2908f7febe7b63349b7e976a46742a48e4144871dcc4a`

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: tensortorrent-0.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Subject digest: f6b3a17d6d32983f860e417fba4b5edc284db38bfa7d2921dfee95a9ea7f9baa
- Sigstore transparency entry: 2397116039
- Sigstore integration time: Aug 9, 2026
Source repository:
- Permalink: alhussein-jamil/TensorTorrent@384b20ae0ec0536e6319965b5addd808086dbe1a
- Branch / Tag: refs/tags/v0.3.1
- Owner: https://github.com/alhussein-jamil
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@384b20ae0ec0536e6319965b5addd808086dbe1a
- Trigger Event: push

File details

Details for the file tensortorrent-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

Download URL: tensortorrent-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Upload date: Aug 9, 2026
Size: 1.5 MB
Tags: CPython 3.10, manylinux: glibc 2.17+ x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tensortorrent-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm	Hash digest
SHA256	`55f2f1e34e10b898064850f446a3f1838a84dac5c2e51b4dbb791a3a0a64bda3`
MD5	`c99ee01359a2bff9cb659533bff0543b`
BLAKE2b-256	`48edba8a5248239bb1f61a72390efe57ab3652064c41a19736f365cf375fc7dc`

See more details on using hashes here.

Provenance

The following attestation bundles were made for tensortorrent-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on alhussein-jamil/TensorTorrent

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: tensortorrent-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Subject digest: 55f2f1e34e10b898064850f446a3f1838a84dac5c2e51b4dbb791a3a0a64bda3
- Sigstore transparency entry: 2397116117
- Sigstore integration time: Aug 9, 2026
Source repository:
- Permalink: alhussein-jamil/TensorTorrent@384b20ae0ec0536e6319965b5addd808086dbe1a
- Branch / Tag: refs/tags/v0.3.1
- Owner: https://github.com/alhussein-jamil
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@384b20ae0ec0536e6319965b5addd808086dbe1a
- Trigger Event: push

Release history Release notifications | RSS feed

0.3.4

Aug 13, 2026

0.3.2

Aug 10, 2026

This release

0.3.1

Aug 9, 2026

0.3.0

Aug 9, 2026

0.2.8

Aug 7, 2026

0.2.7

Aug 7, 2026

0.2.6

Aug 7, 2026

0.2.5

Aug 7, 2026

0.2.4

Aug 6, 2026

0.2.3

Aug 6, 2026

0.2.2

Aug 6, 2026

0.2.1

Aug 6, 2026

0.2.0

Aug 6, 2026

tensortorrent 0.3.1

Headline capacity result (MEASURED)

Install

Quick start

When to use

Docs

License

Project links

Key dates

1 maintainer

Credits

GitHub Statistics

License expression

Requires

Provides Extra

Tags

Classifiers

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

Release history Release notifications | RSS feed