llm-sizer

LLM VRAM calculator + cheapest AWS GPU picker. Size DeepSeek-R1, Llama 70B, Qwen, Mixtral, and other inference workloads before you launch vLLM — then get a single-node instance recommendation and monthly on-demand cost.

Stop guessing whether Llama 3 70B fits on a g5.12xlarge. llm-sizer adds weight VRAM + KV cache VRAM, maps the total to the cheapest AWS GPU that actually fits, and prints a terminal table (or JSON for agents).

pip install llm-sizer
llm-sizer --model deepseek-r1 --quant q4 --concurrent-users 8 --context 32k

Why this exists

Most GPU “rules of thumb” only count model weights. Production vLLM crashes usually come from KV cache growing with context length and concurrent users:

weights     = params × bytes_per_param
kv_cache    = 2 × layers × hidden_size × context × users × bytes_per_param
total_vram  = weights + kv_cache

fp16/bf16 = 2 bytes, q8 = 1 byte, q4 = 0.5 bytes.

Quick start

pip install llm-sizer

llm-sizer --model llama-3-70b --quant fp16 --concurrent-users 4 --context 8k
llm-sizer --model qwen2.5-72b --quant q8 --concurrent-users 16 --context 32k
llm-sizer --list-models

From source:

git clone https://github.com/saheb26/llm-sizer.git
cd llm-sizer
pip install -e .

AI agents / scripts

Machine-readable JSON on stdout (pretty tables stay on the human path):

llm-sizer --model llama-3-70b --quant q4 --context 32k --concurrent-users 8 --json

{
  "tool": "llm-sizer",
  "model": "llama-3-70b",
  "vram_gb": { "weights": 32.88, "kv_cache": 160.0, "total": 192.88 },
  "recommendation": {
    "fits_single_node": true,
    "instance": "g6e.48xlarge",
    "gpu": "8x L40S",
    "vram_gb": 384,
    "usd_per_month": 21694.32
  }
}

Cursor / Claude / Codex: read AGENTS.md and llms.txt.

Example

DeepSeek-R1  |  Q4  |  32k  |  8 users
Weights 312.46 GB  ·  KV cache 106.75 GB  ·  Total 419.21 GB
Recommended: p4de.24xlarge (8x A100 80GB, 640 GB)  ·  $29,495/mo

CLI

Built-in models

DeepSeek-R1 / V3 · Llama 3 / 3.1 / 3.3 (8B, 70B, 405B) · Mistral 7B · Mixtral 8x7B / 8x22B · Qwen2.5 (7B–72B) · Gemma 2 · Phi-4

Aliases accepted: llama70b, llama-70b, deepseek, r1, qwen-72b, mixtral.

AWS catalog

Cheapest us-east-1 on-demand instance whose GPU memory covers total VRAM:

g4dn · g5 · g6 · g6e · p4d · p4de · p5 · p5en

Prices are approximate public list rates and will drift. Treat them as planning numbers, not invoices.

Deploy the architecture

Stop guessing AWS instances and managing vLLM crashes. Deploy this exact architecture securely inside your VPC with one click using CounselNode.

Keywords

LLM VRAM calculator, vLLM GPU memory, KV cache size, DeepSeek-R1 VRAM, Llama 70B AWS instance, cheapest GPU for LLM inference, H100 A100 L40S sizing, AI agent CLI.

License

MIT. Issues and PRs welcome — model cards, newer GPU SKUs, and extra aliases help everyone.