matcha-gpu 0.3.0

GPU energy metering for AI training workloads

matcha

GPU energy metering for AI training workloads

Measure GPU energy consumption of any training run. One command. Zero overhead. Zero code changes.

---

Install

pip install matcha-gpu

Requires an NVIDIA GPU with drivers installed.

Quick Start

Prefix your training command with matcha run:

matcha run torchrun --standalone --nproc_per_node=1 train_gpt.py

Your training runs at full speed. Matcha appends one line at the end:

matcha_energy total:364722J (101.31Wh) duration:746.0s avg_power:489W peak_power:700W samples:7449

That's it. No code changes. No config files. Works with any training script.

How It Works

Matcha runs a lightweight background thread that polls GPU power via NVML at 100ms intervals. Your training process runs natively — no stdout interception, no wrapper overhead. When training finishes, Matcha computes total energy using trapezoidal integration of instantaneous power readings.

Commands

matcha run — Zero overhead energy measurement

# Measure total energy for any training command
matcha run python train.py
matcha run torchrun --standalone --nproc_per_node=1 train_gpt.py
matcha run deepspeed --num_gpus=4 train.py --deepspeed ds_config.json

matcha wrap — Per-step energy breakdown

# See energy for each training step (some overhead)
matcha wrap -p python train.py

Parses stdout for step markers (step 10, iter 10, [10/1000], etc.) and reports energy between each step. Useful for diagnosing energy spikes and inefficient training phases.

matcha monitor — Live GPU power monitoring

# Watch GPU power draw in real time
matcha monitor
matcha monitor --gpu 0 --window 2.0

Python SDK

import matcha

m = matcha.init()

for step in range(num_steps):
    m.step_start()
    # ... your training code, unchanged ...
    energy = m.step_end(step)

summary = m.finish()
# summary.total_energy_j, summary.energy_kwh, summary.j_per_step

Example Output

step:1/20000 train_loss:6.9357 train_time:356ms step_avg:356.10ms
step:2/20000 train_loss:16.7414 train_time:725ms step_avg:362.47ms
...
step:1709/20000 val_loss:2.2111 val_bpb:1.3095 train_time:600097ms step_avg:351.14ms
final_int8_zlib_roundtrip_exact val_loss:2.21311047 val_bpb:1.31072868
matcha_energy total:364722J (101.31Wh) duration:746.0s avg_power:489W peak_power:700W samples:7449

Tested On

• NVIDIA H100 80GB HBM3 — training nanoGPT variants at 500-700W
• Works with torchrun, deepspeed, accelerate, or plain python
• Compatible with PyTorch, JAX, and any framework that runs on NVIDIA GPUs

Why

GPU rental is expensive. Electricity is cheap. But knowing your energy profile tells you whether your GPU is actually working hard or sitting idle — and that directly maps to training efficiency and cost.

10-minute H100 training run:
  Energy cost:   $0.01 (101 Wh @ $0.12/kWh)
  Compute cost:  $0.48 (RunPod @ $2.90/hr)

  → Compute is 48x the energy cost
  → Optimizing energy/step = faster training = less rental time

Roadmap

• Multi-GPU support (aggregate across 8xH100)
• Log file tailing for zero-overhead per-step attribution
• JSONL output for downstream analysis
• Go sidecar binary for production deployments
• Carbon footprint estimation by region

Built by

Keeya Labs · usematcha.dev

License

Apache 2.0