tpu-inference

Pre-release

This release is a pre-release and may not be stable for production use.

vLLM TPU

Latest News 🔥

Pytorch Conference Learn how Spotify uses vLLM with both GPUs and TPUs to drive down costs and improve user experience.
Check back soon for a recording of our session at Ray Summit, November 3-5 in San Francisco!
Check back soon for a recording of our session at JAX DevLab on November 18th in Sunnyvale!
[2025/10] vLLM TPU: A New Unified Backend Supporting PyTorch and JAX on TPU

Previous News 🔥

About

vLLM TPU is now powered by tpu-inference, an expressive and powerful new hardware plugin unifying JAX and PyTorch under a single lowering path within the vLLM project. The new backend now provides a framework for developers to:

Push the limits of TPU hardware performance in open source.
Provide more flexibility to JAX and PyTorch users by running PyTorch model definitions performantly on TPU without any additional code changes, while also extending native support to JAX.
Retain vLLM standardization: keep the same user experience, telemetry, and interface.

Recommended models and features

Although vLLM TPU’s new unified backend makes out-of-the-box high performance serving possible with any model supported in vLLM, the reality is that we're still in the process of implementing a few core components.

For this reason, we’ve provided a Recommended Models and Features page detailing the models and features that are validated through unit, integration, and performance testing.

Get started

Get started with vLLM on TPUs by following the quickstart guide.

Visit our documentation to learn more.

Compatible TPU Generations

Recommended: v7x, v5e, v6e
Experimental: v3, v4, v5p

Recipes

Contribute

We're always looking for ways to partner with the community to accelerate vLLM TPU development. If you're interested in contributing to this effort, check out the Contributing guide and Issues to start. We recommend filtering Issues on the good first issue tag if it's your first time contributing.

Contact us

For technical questions and feature requests, open a GitHub Issue
For feature requests, please open one on Github here
For discussing with fellow users, use the TPU support topic in the vLLM Forum
For coordinating contributions and development, use the Developer Slack
For collaborations and partnerships, contact us at vllm-tpu@google.com

Metadata

Release files for tpu-inference 0.13.2.dev20260120

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tpu-inference 0.13.2.dev20260120
File	Size	Uploaded
tpu_inference-0.13.2.dev20260120.tar.gz	521.0 kB	Jan 20, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for tpu-inference 0.13.2.dev20260120
File	Interpreter	ABI	Platform	Reset
tpu_inference-0.13.2.dev20260120-py3-none-any.whl 710.5 kB Jan 20, 2026	Python 3	none	any	Details

Total release size: 1.2 MB

Release files / tpu_inference-0.13.2.dev20260120.tar.gz

Download URL	tpu_inference-0.13.2.dev20260120.tar.gz
Size	521.0 kB
Tags	Source
SHA-256 checksum How to use checksums	`6e94c603305505a839d5b5be33d5bda146ac7e987abfe2fdf2974bbe65c6a08e`
BLAKE2b-256 checksum How to use checksums	`b2fbc6415564d3325466a60897f42e8308d601d8a057b3b3614e377db4da7dd5`
Upload date	Jan 20, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.13.7`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/vllm-project/tpu-inference
Publishing commit: github.com/vllm-project/tpu-inference/tree/723daa6ea24016e2e737400483ae91c4c2424c6e

Workflow

Publishing configuration: github.com/vllm-project/tpu-inference/blob/723daa6ea24016e2e737400483ae91c4c2424c6e/.github/workflows/release.yml
Publishing logs: github.com/vllm-project/tpu-inference/actions/runs/21164252005/attempts/1

Artifact

Subject: tpu_inference-0.13.2.dev20260120.tar.gz
SHA-256 checksum: 6e94c603305505a839d5b5be33d5bda146ac7e987abfe2fdf2974bbe65c6a08e Verifying attestations

Release files / tpu_inference-0.13.2.dev20260120-py3-none-any.whl

Download URL	tpu_inference-0.13.2.dev20260120-py3-none-any.whl
Size	710.5 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`c4a93bd574e265ffa82f5739674f591f8838ea0e41874724ad24a8d534a97918`
BLAKE2b-256 checksum How to use checksums	`47c6747379b3aacf5a0a917eaf46a67020b7185232190b3359bd2ea5910f1a19`
Upload date	Jan 20, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.13.7`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/vllm-project/tpu-inference
Publishing commit: github.com/vllm-project/tpu-inference/tree/723daa6ea24016e2e737400483ae91c4c2424c6e

Workflow

Publishing configuration: github.com/vllm-project/tpu-inference/blob/723daa6ea24016e2e737400483ae91c4c2424c6e/.github/workflows/release.yml
Publishing logs: github.com/vllm-project/tpu-inference/actions/runs/21164252005/attempts/1

Artifact

Subject: tpu_inference-0.13.2.dev20260120-py3-none-any.whl
SHA-256 checksum: c4a93bd574e265ffa82f5739674f591f8838ea0e41874724ad24a8d534a97918 Verifying attestations