hfit 0.1.0

Check whether a Hugging Face model fits your local hardware — without downloading the weights.

hfit

Hugging Face + fit.

Will this Hugging Face model run on my machine? Find out in seconds without downloading a single weight file.

Give it a model id like Qwen/Qwen2.5-7B-Instruct. It reads the exact parameter count and architecture from the Hub's metadata (a few KB of API calls), detects your local hardware (GPU VRAM, RAM), and tells you which precisions fit:

┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Precision           ┃  Weights ┃ KV cache ┃ Required* ┃ Verdict               ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━┩
│ FP16 / BF16 (native)│ 14.19 GB │  0.22 GB │  15.91 GB │ ◐ CPU/RAM only (slow) │
│ INT8 (quantized)    │  7.09 GB │  0.22 GB │   8.47 GB │ ◐ CPU/RAM only (slow) │
│ INT4 (quantized)    │  3.55 GB │  0.22 GB │   4.74 GB │ ● tight on GPU        │
└─────────────────────┴──────────┴──────────┴───────────┴───────────────────────┘

Unlike accelerate estimate-memory, this compares the model against your actual hardware and includes a KV-cache estimate at your chosen context length computed exactly from the model's architecture (layers × KV heads × head dim), not a flat percentage.

Install

pip install hfit
# or from a clone of this repo:
pip install .

Dependencies: huggingface_hub, rich, textual. No torch, no CUDA needed.

Usage

hfit                                  # interactive TUI
hfit Qwen/Qwen2.5-7B-Instruct         # one-shot report
hfit meta-llama/Llama-3.1-8B-Instruct --ctx 32768
hfit some/gated-model --token hf_xxx

In the TUI: type a model id, press Enter; switch the context length from the dropdown to see the KV cache impact instantly (no refetch). q quits.

Exit codes for scripting: 0 = fits somewhere (GPU or RAM), 1 = lookup error, 3 = does not fit at all.

What it understands

• Safetensors repos — exact parameter count from Hub metadata.
• GGUF repos (e.g. bartowski/...-GGUF) every quant file assessed individually, for llama.cpp / Ollama / LM Studio users.
• Multi-component repos (speech/vision pipelines with several weight folders) reported as a stored total instead of a fake parameter count.
• Duplicate serializations (safetensors + bin + h5, sharded + consolidated) are deduplicated, not double-counted.

Platform support

Platform	GPU detection	RAM detection
Linux	nvidia-smi; AMD via sysfs (no ROCm needed), then rocm-smi	/proc/meminfo
Windows	nvidia-smi (PATH or System32)	WinAPI via ctypes
macOS (Apple Silicon)	unified memory (RAM = VRAM budget)	sysctl

psutil is used when available (pip install .[full]) but is not required.

How the estimate works

required = weights + KV cache(context) + overhead(0.8 GB + 5% of weights)

• Weights: parameters × bytes-per-parameter (4 / 2 / 1 / 0.5 for FP32 / FP16 / INT8 / INT4).
• KV cache: 2 × layers × kv_heads × head_dim × 2 bytes × context_tokens from config.json; a 20% margin is used when the architecture is unknown.
• Verdicts leave ~7% VRAM and ~15% RAM headroom for the driver and OS.

These are pre-flight estimates. For a definitive runtime answer, vLLM's --dry-run on the actual machine remains the ground truth.