smelt

This-host GPU efficiency profiler for vLLM. One model, one server.

Objective: minimize $/1M tokens subject to your p95 latency SLO.

You already know the GPU-hour price. Smelt turns a live nvidia-smi + vLLM /metrics sample into $/hour → tok/s → $/1M tokens, names waste it can measure, and tells you whether the last flag change won.

pip install smelt-gpu && smelt analyze --demo --gpu-hourly 3.89
make run                       # this repo: install from PyPI + scan/analyze/optimize
make install                   # editable checkout + dashboard
smelt scan --vllm-url http://127.0.0.1:8000
smelt analyze --gpu-hourly 3.89
smelt experiment save --name before
# change one vLLM flag, reload
smelt experiment compare --baseline before --slo-ms 500
make dev                       # API :8741 + dashboard :3010

PyPI name is smelt-gpu; the CLI stays smelt. smelt --version is the version.

Smelt does not restart vLLM or edit its flags. You apply the change; it records and compares. Keep the candidate only if $/1M dropped (or same cost, more tok/s) and p95 still meets the SLO.

If nothing is live, analyze falls back to a sample fleet (96×H100 + 32×H200). --demo forces that.

The wheel is the CLI + smelt serve. The Next.js dashboard is this repo only (make dev). Full notes: docs/.

Use it next to your vLLM

pip install smelt-gpu
smelt analyze --vllm-url http://127.0.0.1:8000 --gpu-hourly 3.89

In Python:

from smelt.integrations.scan import collect_host
from smelt.snapshot import fleet_from_snapshot, host_summary

snap = collect_host(vllm_url="http://127.0.0.1:8000")
fleet = fleet_from_snapshot(snap, gpu_hourly=3.89)
print(host_summary(fleet))

Or smelt serve and GET /api/fleet?live=true. How to wire this into a job or another service: docs/use.md.

Improve it

make install && make hooks
make test
make test-live          # this GPU + a server
make verify

New GPU prices → classify_gpu / GPU_HOURLY_USD. New /metrics fields → smelt/integrations/vllm.py plus a fixture under tests/fixtures/. Waste rules → _opportunities() in snapshot.py. Details: docs/use.md.