One screen for running and fine-tuning AI models on an Apple Silicon Mac, or on Linux with NVIDIA GPUs (experimental): how fast the model is (or how well it is learning), how close it is to running out of memory, and how hot the machine is getting, on the same timeline.

Activity Monitor (or nvtop) makes you switch views and knows nothing about models. Benchmark apps time the requests they send themselves. inference-vitals sits beside whatever you are running and records both sides, the model and the machine, so you can see why a run slowed down: the GPU got hot, the system started swapping, or a page was simply longer.

What you get

• Model runs, with any local engine (Ollama, LM Studio, llama.cpp, the MLX servers, Jan, LocalAI) or your own code: speed live while the answer is written, time to first token, context used, answers cut off or invalid.
• Fine-tunes: training and validation loss, steps and time left, and whether the model is still learning or has started memorising its examples.
• The machine: GPU memory left, memory pressure, swap, GPU use, temperatures, thermal state, power.
• Energy: what the machine used over any range, how much of it was the AI, and the cost at your rate.
• Battery on a laptop: the charge, and on battery how long it will last at this load, with a warning when it will not outlast the run.

It fits one screen: the live numbers stay in view while tabs swap the charts below them, and you star the charts you want together in Favourites. Every tile and tab: docs/dashboard.md.

Quick start

On an Apple Silicon Mac with Homebrew:

brew install uv prometheus macmon
uv tool install inference-vitals
inference-vitals start
open http://127.0.0.1:3000

The machine's numbers appear straight away. Linux with NVIDIA, Grafana, troubleshooting: docs/install.md.

Connect what you run

Privacy

Everything runs on your machine and listens on 127.0.0.1 only; the page loads nothing from the internet; only numbers are collected, never prompts, answers or file names. More: docs/limits.md.

Documentation

It is Prometheus with a dashboard page of its own (or Grafana), macmon for the chip on a Mac (on Linux, a small exporter that reads nvidia-smi), small exporters for memory and the battery, the proxy, and a tiny Python library, with no dependencies beyond Python itself.

Credits

inference-vitals runs macmon (MIT) and Prometheus (Apache-2.0), and optionally Grafana (AGPL-3.0), installed separately (Homebrew on a Mac, your package manager on Linux); none of them is bundled here.

Licence

MIT. See LICENSE.