tpu-info CLI
tpu-info is a simple CLI tool for detecting Cloud TPU devices and reading
runtime metrics from libtpu, including memory usage and duty cycle. It
supports both a static, one-time snapshot and a live streaming mode to monitor
metrics continuously.
Note: to access libtpu utilization metrics, you must have a workload
running with a supported ML framework, such as JAX or PyTorch/XLA. See the
Usage section for more information.
What's New in Version 0.14.2
🚀 New Features
- Add support for Pygrain metrics from the local Prometheus server
- Add support for Orbax performance and operational metrics from the local Prometheus server
🛠️ Bug Fixes
- [0.14.2] Fix fatal check failure when importing
tpu_infodirectly - [0.14.1] Fix ZeroDivisionError in HLO execution timing metrics calculation when the TPU is idle
- [0.14.1] Fix potential Rich markup rendering injection vulnerabilities where malicious telemetry data or error messages could trigger unexpected styles or cause crashes
Installing
Install the latest release using pip:
pip install tpu-info
Alternatively, install tpu-info from source:
pip install git+https://github.com/AI-Hypercomputer/cloud-accelerator-diagnostics/#subdirectory=tpu_info
Usage
To view current TPU utilization data, tpu-info requires a running TPU workload
with supported ML framework1 such as JAX or PyTorch/XLA. For example:
# JAX
>>> import jax
>>> jax.device_count()
4
# Create a tensor on the TPU
>>> t = jax.numpy.ones((300, 300))
# PyTorch/XLA
>>> import torch
>>> import torch_xla
>>> t = torch.randn((300, 300), device=torch_xla.device())
Then, on the same machine, you can run the tpu-info command in your terminal.
Static Mode
Run the following command for a one-time snapshot of the current metrics.
$ tpu-info
Libtpu version: 0.0.19.dev20250721+nightly
Accelerator type: v6e
TPU Chips
| Chip | Type | Devices | PID |
|-------------|--------------|---------|---------|
| /dev/vfio/0 | TPU v6e chip | 1 | 1469584 |
| /dev/vfio/1 | TPU v6e chip | 1 | 1469584 |
| /dev/vfio/2 | TPU v6e chip | 1 | 1469584 |
| /dev/vfio/3 | TPU v6e chip | 1 | 1469584 |
TPU Runtime Utilization
| Chip | HBM Usage (GiB) | Duty cycle |
|------|-----------------------|------------|
| 0 | 18.45 GiB / 31.25 GiB | 100.00% |
| 1 | 10.40 GiB / 31.25 GiB | 100.00% |
| 2 | 10.40 GiB / 31.25 GiB | 100.00% |
| 3 | 10.40 GiB / 31.25 GiB | 100.00% |
TensorCore Utilization
| Core ID | TensorCore Utilization |
|---------|------------------------|
| 0 | 13.60% |
| 1 | 14.81% |
| 2 | 14.36% |
| 3 | 13.60% |
TPU Buffer Transfer Latency
| Buffer Size | P50 | P90 | P95 | P999 |
|--------------|--------------|--------------|--------------|--------------|
| 8MB+ | 108978.82 us | 164849.81 us | 177366.42 us | 212419.07 us |
| 4MB+ | 21739.38 us | 38126.84 us | 42110.12 us | 55474.21 us |
TPU Inbound Buffer Transfer Latency
| Buffer Size | P50 | P90 | P95 | P999 |
|--------------|--------------|--------------|--------------|--------------|
| 8MB+ | 18945.59 us | 34461.46 us | 39652.74 us | 56051.94 us |
| 4MB+ | 4829.09 us | 8594.43 us | 10236.53 us | 17754.86 us |
TPU Host Compute Latency
| Buffer Size | P50 | P90 | P95 | P999 |
|--------------|--------------|--------------|--------------|--------------|
| 8MB+ | 998.17 us | 3605.34 us | 6292.10 us | 11608.01 us |
| 4MB+ | 678.33 us | 2611.93 us | 5258.30 us | 11083.23 us |
TPU gRPC TCP Minimum RTT
| P50 | P90 | P95 | P999 |
|----------|----------|----------|----------|
| 35.99 us | 52.15 us | 53.83 us | 55.51 us |
TPU gRPC TCP Delivery Rate
| P50 | P90 | P95 | P999 |
|---------------|---------------|---------------|---------------|
| 12305.96 Mbps | 18367.10 Mbps | 24872.11 Mbps | 44841.55 Mbps |
Streaming Mode
You can run tpu-info in a streaming mode to periodically refresh and display
the utilization statistics.
# Refresh stats every 2 seconds
tpu-info --streaming --rate 2
Refresh rate: 0.1s
Last update: 2025-07-24 11:00:59 UTC
Libtpu version: 0.0.19.dev20250721+nightly
Accelerator type: v6e
TPU Chips
| Chip | Type | Devices | PID |
|--------------|--------------|---------|--------|
| /dev/vfio/0 | TPU v6e chip | 1 | 1022 |
| /dev/vfio/1 | TPU v6e chip | 1 | 1022 |
| /dev/vfio/2 | TPU v6e chip | 1 | 1022 |
| /dev/vfio/3 | TPU v6e chip | 1 | 1022 |
TPU Runtime Utilization
| Chip | HBM Usage (GiB) | Duty cycle |
|--------|--------------------------|------------|
| 8 | 17.26 GiB / 31.25 GiB | 100.00% |
| 9 | 9.26 GiB / 31.25 GiB | 100.00% |
| 12 | 9.26 GiB / 31.25 GiB | 100.00% |
| 13 | 9.26 GiB / 31.25 GiB | 100.00% |
TensorCore Utilization
| Core ID | TensorCore Utilization |
|---------|------------------------|
| 0 | 15.17% |
| 1 | 14.62% |
| 2 | 14.68% |
| 3 | 15.14% |
TPU Buffer Transfer Latency
| Buffer Size | P50 | P90 | P95 | P999 |
|--------------|--------------|--------------|--------------|--------------|
| 8MB+ | 18264.03 us | 33263.06 us | 35990.98 us | 53997.32 us |
TPU Inbound Buffer Transfer Latency
| Buffer Size | P50 | P90 | P95 | P999 |
|--------------|--------------|--------------|--------------|--------------|
| 8MB+ | 18316.95 us | 32857.03 us | 36501.59 us | 58854.54 us |
TPU Host Compute Latency
| Buffer Size | P50 | P90 | P95 | P999 |
|--------------|--------------|--------------|--------------|--------------|
| 8MB+ | 678.33 us | 2611.93 us | 5258.30 us | 11083.23 us |
TPU gRPC TCP Minimum RTT
| P50 | P90 | P95 | P999 |
|----------|----------|----------|----------|
| 35.99 us | 52.15 us | 53.83 us | 55.51 us |
TPU gRPC TCP Delivery Rate
| P50 | P90 | P95 | P999 |
|---------------|---------------|---------------|---------------|
| 12305.96 Mbps | 18367.10 Mbps | 24872.11 Mbps | 44841.55 Mbps |
Version
To check the installed version of tpu-info, libtpu version and accelerator
type of the TPU chip, use the --version or -v flag.
Compatible Environment:
$ tpu-info --version
- tpu-info version: 0.8.0
- libtpu version: 0.0.18
- accelerator type: v6e
Incompatible Environment (Python 3.12+):
$ tpu-info --version
- tpu-info version: 0.8.0
- libtpu version: N/A (incompatible environment)
- accelerator type: N/A (incompatible environment)
Process
You can use the --process or -p flag to display information about the
processes currently running on the TPU.
$ tpu-info --process
TPU Process Info
| Chip | PID | Process Name |
|-------------|--------|--------------|
| /dev/vfio/0 | 799657 | python3 |
| /dev/vfio/1 | 799657 | python3 |
| /dev/vfio/2 | 799657 | python3 |
| /dev/vfio/3 | 799657 | python3 |
| /dev/vfio/4 | 799657 | python3 |
| /dev/vfio/5 | 799657 | python3 |
| /dev/vfio/6 | 799657 | python3 |
| /dev/vfio/7 | 799657 | python3 |
List Metrics
You can use the --list_metrics flag to display all supported metrics that can
be given along with the --metric flag.
$ tpu-info --list_metrics
╭─ Supported Metrics ─────────────────────────────────────────────────────────────────────────────╮
│ buffer_transfer_latency │
│ checkpoint_write_async_blocking_duration │
│ checkpoint_write_async_commit_duration │
│ checkpoint_write_async_commit_future_count │
│ checkpoint_write_async_metadata_write_duration │
│ checkpoint_write_async_ocdbt_merge_duration │
│ checkpoint_write_async_thread_duration │
│ checkpoint_write_async_total_duration │
│ checkpoint_write_duration_since_last_checkpoint │
│ checkpoint_write_get_old_steps_duration │
│ checkpoint_write_old_steps_examined_count │
│ checkpoint_write_preemption_duration_saved │
│ checkpoint_write_remove_steps_duration │
│ checkpoint_write_wait_for_previous_duration │
│ collective_e2e_latency │
│ core_state │
│ device_to_host_transfer_latency │
│ duty_cycle_percent │
│ grpc_tcp_delivery_rate │
│ grpc_tcp_min_rtt │
│ hbm_usage │
│ hlo_exec_timing │
│ hlo_queue_size │
│ host_compute_latency │
│ host_to_device_transfer_latency │
│ inbound_buffer_transfer_latency │
│ orbax_read_async_blocking_duration │
│ orbax_read_async_start_count │
│ orbax_read_start_count │
│ orbax_read_storage_type │
│ orbax_read_total_duration │
│ orbax_read_worker_io_requested_size │
│ orbax_read_worker_io_requested_throughput │
│ orbax_write_async_directory_creation_duration │
│ orbax_write_async_start_count │
│ orbax_write_blocking_throughput │
│ orbax_write_checkpoint_start_sync_duration │
│ orbax_write_directory_creation_duration │
│ orbax_write_preemption_count │
│ orbax_write_replicated_array_size │
│ orbax_write_sharded_array_size │
│ orbax_write_size │
│ orbax_write_start_count │
│ orbax_write_storage_type │
│ orbax_write_success_count │
│ orbax_write_throughput │
│ pygrain_data_sources_read_size │
│ pygrain_dataloader_iterator_get_next_duration │
│ pygrain_dataset_next_duration │
│ pygrain_dataset_prefetch_buffer_ready_count │
│ pygrain_dataset_source_read_duration │
│ queued_programs │
│ runtime_hbm_utilization │
│ sequencer_state │
│ sequencer_state_detailed │
│ tensorcore_idle_duration │
│ tensorcore_utilization │
╰─────────────────────────────────────────────────────────────────────────────────────────────────╯
Metric
You can use the --metric flag to display specific metrics. You can specify
multiple metrics separated by spaces with multiple --metric flags.
$ tpu-info --metric duty_cycle_percent --metric hbm_usage
TPU Duty Cycle
| Core ID | Duty Cycle (%) |
|---------|----------------|
| 0 | 100.00% |
| 1 | 100.00% |
| 2 | 100.00% |
| 3 | 100.00% |
| 4 | 100.00% |
| 5 | 100.00% |
| 6 | 100.00% |
| 7 | 100.00% |
TPU HBM Usage
| Chip | HBM Usage (GiB) |
|--------|-----------------------|
| 0 | 29.50 GiB / 31.25 GiB |
| 1 | 21.50 GiB / 31.25 GiB |
| 2 | 21.50 GiB / 31.25 GiB |
| 3 | 21.50 GiB / 31.25 GiB |
| 4 | 21.50 GiB / 31.25 GiB |
| 5 | 21.50 GiB / 31.25 GiB |
| 6 | 21.50 GiB / 31.25 GiB |
| 7 | 21.50 GiB / 31.25 GiB |
-
Releases from before 2024 may not be compatible. ↩
Metadata
Release files for tpu-info 0.14.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| tpu_info-0.14.2-py3-none-any.whl | Python 3 | none | any | Details for tpu_info-0.14.2-py3-none-any.whl |
Release files / tpu_info-0.14.2-py3-none-any.whl
| Download URL | tpu_info-0.14.2-py3-none-any.whl |
|---|---|
| Size | 44.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9e11ccdb4a64b9c6858311a560c992546224a1fa09da660b7b63983af17c4196
|
|
BLAKE2b-256 checksum How to use checksums |
dadc9e8fc37cfa42e8cb8a9225e57d4c50be668c7ff0a07560ba27c6357f7cce
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.9
|
Security for tpu-info 0.14.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.