smelt

This-host GPU efficiency profiler for vLLM. One model, one server.

Objective: minimize $/1M tokens subject to your p95 latency SLO.

You already know the GPU-hour price. Smelt turns a live nvidia-smi + vLLM /metrics sample into $/hour → tok/s → $/1M tokens, names waste it can measure, and tells you whether the last flag change won.

make install                   # this repo
smelt scan --vllm-url http://127.0.0.1:8000
smelt analyze --gpu-hourly 3.89
smelt experiment save --name before
# change one vLLM flag, reload
smelt experiment compare --baseline before --slo-ms 500
make dev                       # API :8741 + dashboard :3010

pip install smelt-gpu is the intended PyPI name (CLI stays smelt). It is not published yet — use this repo or make package and install the wheel. smelt --version is the version.

Smelt does not restart vLLM or edit its flags. You apply the change; it records and compares. Keep the candidate only if $/1M dropped (or same cost, more tok/s) and p95 still meets the SLO.

If nothing is live, analyze falls back to a sample fleet (96×H100 + 32×H200). --demo forces that.

The wheel is the CLI + smelt serve. The Next.js dashboard is this repo only (make dev). Full notes: docs/.