Skip to main content

vram-fit

Stop guessing which local LLM fits your GPU. Scan VRAM, calculate GGUF + KV-cache math, and get exact Ollama & llama.cpp commands.

Python 3.11+ License: MIT Agent Skill Slop Score: 0%

VRAM Calculation Architecture


Quick Start

Install via pip:

pip install llm-vram-fit

Or run directly without installation:

uvx llm-vram-fit fit

Install as an Agent Skill for coding assistants (Claude Code, Cursor, Codex):

npx skills add an1zy/vram-fit

Features

  • Hardware Detection: Automatically detects NVIDIA GPUs via nvidia-smi, Apple Silicon unified memory via macOS metrics, or CPU-only profiles with system RAM inspection.
  • Accurate GQA Memory Formula: Computes exact Grouped-Query Attention (GQA) KV-cache footprints across context lengths (4k to 128k) and KV data types (fp16, q8_0, q4_0).
  • GGUF Weight Math: Pre-calculated Bits-Per-Weight (BPW) tables for IQ4_XS, Q4_K_M, Q5_K_M, Q6_K, and Q8_0 with metadata overhead factors.
  • Offload Status Categorization: Classifies models into FULL_GPU (100% VRAM), HYBRID (partial GPU offloading), or OOM (out of memory).
  • Execution Ready Commands: Emits copy-pasteable environment variables and flags for PowerShell, Bash, Ollama, and llama.cpp llama-server.
  • Benchmark Utility: Built-in latency and tokens-per-second (prompt_tps, gen_tps) benchmarking against local Ollama instances.

CLI Usage

1. Fit Analysis

Analyze memory fit for curated models on current hardware:

vram-fit fit

Filter by category (coding, reasoning, general) or view only models that fit 100% in VRAM:

vram-fit fit --category coding --only-gpu

vram-fit fit demo

2. Hardware Simulation

Simulate target hardware setups before purchasing GPUs or renting cloud instances:

vram-fit fit --vram 24 --ram 64 --quant Q5_K_M --ctx 16384

3. Generate Launch Commands

Generate production launch commands for a specific model ID or Ollama tag:

vram-fit cmd qwen2.5-coder-7b --vram 8 --ctx 8192 --kv q8_0

vram-fit cmd demo

4. Hardware Scan

Inspect host GPU VRAM, system RAM, CPU cores, and unified memory configuration:

vram-fit scan

5. Benchmark Ollama

Benchmark prompt evaluation and token generation speed on a running Ollama server:

vram-fit bench qwen2.5-coder:7b --url http://127.0.0.1:11434

Agent and Machine-Readable JSON Mode

Every command supports --json for integration with autonomous agents, workflows, and automated pipelines:

vram-fit scan --json
vram-fit fit --only-gpu --json

Example JSON payload:

{
  "hardware": {
    "gpu_name": "NVIDIA GeForce RTX 3060 Ti",
    "gpu_vendor": "NVIDIA",
    "vram_total_gb": 8.0,
    "vram_free_gb": 5.23,
    "ram_total_gb": 31.95,
    "ram_free_gb": 17.09,
    "cpu_cores": 8,
    "unified_memory": false
  },
  "results": [
    {
      "model_id": "qwen2.5-coder-7b",
      "model_name": "Qwen 2.5 Coder 7B",
      "category": "coding",
      "status": "FULL_GPU",
      "weights_gb": 4.84,
      "kv_cache_gb": 0.23,
      "total_gb": 5.57,
      "gpu_layers_pct": 100.0,
      "max_ctx_full_gpu": 32768,
      "ollama_cmd": "OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama run qwen2.5-coder:7b",
      "llama_cpp_cmd": "llama-cli -m qwen2.5-coder-7b.gguf -ngl 28 -c 8192 -ctk q8_0 -ctv q8_0 --flash-attn"
    }
  ]
}

Model Catalog

The built-in catalog includes 10 popular local models across size categories:

  • 4B–9B: gemma3:4b, qwen2.5-coder:7b, qwen3:8b, deepseek-r1:8b
  • 12B–14B: gemma3:12b, phi4:14b, qwen2.5-coder:14b
  • MoE: qwen3:30b-a3b (Active: 3.3B)
  • 27B–32B: gemma3:27b, qwen2.5-coder:32b

License

MIT © 2026 an1zy

Metadata

Release files for llm-vram-fit 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for llm-vram-fit 0.1.0
File Size Uploaded Provenance Actions
llm_vram_fit-0.1.0.tar.gz 10.7 kB With provenance Details for llm_vram_fit-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for llm-vram-fit 0.1.0
File Interpreter ABI Platform Actions
llm_vram_fit-0.1.0-py3-none-any.whl Python 3 none any Details for llm_vram_fit-0.1.0-py3-none-any.whl

Total release size: 22.4 kB

Release files / llm_vram_fit-0.1.0.tar.gz

Download URL llm_vram_fit-0.1.0.tar.gz
Size 10.7 kB
Tags Source
SHA-256 checksum
How to use checksums
a8794be15e9178a795a6e7c7aac0211cac35349436d56f5dd258a1267c803e8b
BLAKE2b-256 checksum
How to use checksums
3fdf0daf46477ab4583f1f8f670fe61cad1c06a6c26c0fdcc83a473db8313d35
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.

Transparency log

Release files / llm_vram_fit-0.1.0-py3-none-any.whl

Download URL llm_vram_fit-0.1.0-py3-none-any.whl
Size 11.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
dc88508696788719347cf5476482da04ef3355703d5ffabeed6c53cf6adcbb0d
BLAKE2b-256 checksum
How to use checksums
9a5cca39d5676dc7208b49ec527d3be3179756231b9cc543092031e8125a456b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.

Transparency log

Security for llm-vram-fit 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for llm_vram_fit-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page