vram-fit
Stop guessing which local LLM fits your GPU. Scan VRAM, calculate GGUF + KV-cache math, and get exact Ollama & llama.cpp commands.
Quick Start
Install via pip:
pip install llm-vram-fit
Or run directly without installation:
uvx llm-vram-fit fit
Install as an Agent Skill for coding assistants (Claude Code, Cursor, Codex):
npx skills add an1zy/vram-fit
Features
- Hardware Detection: Automatically detects NVIDIA GPUs via
nvidia-smi, Apple Silicon unified memory via macOS metrics, or CPU-only profiles with system RAM inspection. - Accurate GQA Memory Formula: Computes exact Grouped-Query Attention (GQA) KV-cache footprints across context lengths (4k to 128k) and KV data types (
fp16,q8_0,q4_0). - GGUF Weight Math: Pre-calculated Bits-Per-Weight (BPW) tables for
IQ4_XS,Q4_K_M,Q5_K_M,Q6_K, andQ8_0with metadata overhead factors. - Offload Status Categorization: Classifies models into
FULL_GPU(100% VRAM),HYBRID(partial GPU offloading), orOOM(out of memory). - Execution Ready Commands: Emits copy-pasteable environment variables and flags for PowerShell, Bash, Ollama, and llama.cpp
llama-server. - Benchmark Utility: Built-in latency and tokens-per-second (
prompt_tps,gen_tps) benchmarking against local Ollama instances.
CLI Usage
1. Fit Analysis
Analyze memory fit for curated models on current hardware:
vram-fit fit
Filter by category (coding, reasoning, general) or view only models that fit 100% in VRAM:
vram-fit fit --category coding --only-gpu
2. Hardware Simulation
Simulate target hardware setups before purchasing GPUs or renting cloud instances:
vram-fit fit --vram 24 --ram 64 --quant Q5_K_M --ctx 16384
3. Generate Launch Commands
Generate production launch commands for a specific model ID or Ollama tag:
vram-fit cmd qwen2.5-coder-7b --vram 8 --ctx 8192 --kv q8_0
4. Hardware Scan
Inspect host GPU VRAM, system RAM, CPU cores, and unified memory configuration:
vram-fit scan
5. Benchmark Ollama
Benchmark prompt evaluation and token generation speed on a running Ollama server:
vram-fit bench qwen2.5-coder:7b --url http://127.0.0.1:11434
Agent and Machine-Readable JSON Mode
Every command supports --json for integration with autonomous agents, workflows, and automated pipelines:
vram-fit scan --json
vram-fit fit --only-gpu --json
Example JSON payload:
{
"hardware": {
"gpu_name": "NVIDIA GeForce RTX 3060 Ti",
"gpu_vendor": "NVIDIA",
"vram_total_gb": 8.0,
"vram_free_gb": 5.23,
"ram_total_gb": 31.95,
"ram_free_gb": 17.09,
"cpu_cores": 8,
"unified_memory": false
},
"results": [
{
"model_id": "qwen2.5-coder-7b",
"model_name": "Qwen 2.5 Coder 7B",
"category": "coding",
"status": "FULL_GPU",
"weights_gb": 4.84,
"kv_cache_gb": 0.23,
"total_gb": 5.57,
"gpu_layers_pct": 100.0,
"max_ctx_full_gpu": 32768,
"ollama_cmd": "OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama run qwen2.5-coder:7b",
"llama_cpp_cmd": "llama-cli -m qwen2.5-coder-7b.gguf -ngl 28 -c 8192 -ctk q8_0 -ctv q8_0 --flash-attn"
}
]
}
Model Catalog
The built-in catalog includes 10 popular local models across size categories:
- 4B–9B:
gemma3:4b,qwen2.5-coder:7b,qwen3:8b,deepseek-r1:8b - 12B–14B:
gemma3:12b,phi4:14b,qwen2.5-coder:14b - MoE:
qwen3:30b-a3b(Active: 3.3B) - 27B–32B:
gemma3:27b,qwen2.5-coder:32b
License
MIT © 2026 an1zy
Metadata
Release files for llm-vram-fit 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| llm_vram_fit-0.1.0.tar.gz | 10.7 kB | With provenance | Details for llm_vram_fit-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| llm_vram_fit-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for llm_vram_fit-0.1.0-py3-none-any.whl |
Total release size: 22.4 kB
Release files / llm_vram_fit-0.1.0.tar.gz
| Download URL | llm_vram_fit-0.1.0.tar.gz |
|---|---|
| Size | 10.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a8794be15e9178a795a6e7c7aac0211cac35349436d56f5dd258a1267c803e8b
|
|
BLAKE2b-256 checksum How to use checksums |
3fdf0daf46477ab4583f1f8f670fe61cad1c06a6c26c0fdcc83a473db8313d35
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.
Transparency logRelease files / llm_vram_fit-0.1.0-py3-none-any.whl
| Download URL | llm_vram_fit-0.1.0-py3-none-any.whl |
|---|---|
| Size | 11.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
dc88508696788719347cf5476482da04ef3355703d5ffabeed6c53cf6adcbb0d
|
|
BLAKE2b-256 checksum How to use checksums |
9a5cca39d5676dc7208b49ec527d3be3179756231b9cc543092031e8125a456b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.
Transparency logSecurity for llm-vram-fit 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for llm_vram_fit-0.1.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.