FitCheck

Estimate LLM memory before you start a GPU job.

What FitCheck does

FitCheck estimates whether a LoRA, QLoRA, or full fine-tuning configuration is likely to fit on a selected GPU before training starts. It reads the model's Hugging Face config.json and parameter-count metadata; it never downloads model weights, imports PyTorch, or needs CUDA at estimate time.

The estimate includes base weights, LoRA adapters, optimizer states, gradients, activations, and CUDA/allocator overhead. The result is a MiB total, a component breakdown, a fit verdict, and the largest micro-batch that fits. An inference mode also estimates resident weights and KV-cache memory.

Who it is for

FitCheck is for ML students, researchers, and startup engineers choosing a GPU or checking a training job in CI. It helps catch an obvious OOM before a long download, a paid GPU session, or a teammate's time is wasted.

It is a sizing tool, not a throughput benchmark or a guarantee that an unmeasured workload will not OOM.

Quick start

pip install fitcheck-llm

fitcheck NousResearch/Meta-Llama-3.1-8B --qlora --lora-r 64 --batch-size 4 --seq-len 2048 --optimizer adamw --flash-attn --gpu 4090

The runtime uses Click, Rich, and huggingface-hub; it does not require torch or CUDA. Gated Hugging Face models still need normal Hub access, for example hf auth login.

What you can do

FitCheck supports four related workflows:

The training, inference, and advisor commands support --json for CI/CD. Training also supports --explain for a plain-English breakdown and savings hints, --verbose for the per-layer activation detail, --list-gpus to inspect the GPU database, and --vram-mib for a custom usable-memory budget.

Inside the interactive session, the available tasks are:

What the output means

Peak VRAM = weights + LoRA + optimizer + gradients + activations + overhead

• The component table shows what is using the memory.
• The verdict compares the estimate with the selected GPU's usable MiB.
• max micro-batch is found by running the full estimator at different batch sizes, not by dividing free memory by one component.
• UNCERTAIN means the point estimate fits, but the measured safety reserve is not large enough to call it safe.
• In CLI mode, exit code 0 means the request fits, 1 means it does not fit or is uncertain, and 2 means the estimate could not run. For advise, 0 means at least one swept configuration fits. The REPL keeps verdicts inside the session and exits 0.

--precision is the compute dtype. --quant is the base-model storage format. They are separate settings: QLoRA, for example, uses an NF4 base with BF16 compute.

Accuracy evidence

The beta evidence is measured and reproducible, not a universal accuracy promise. The archive contains 79 archived runs: 49 fitted, 57 calibration/repeat rows scored, and 10 legacy rows excluded from fitting. The project ships four T4 profiles for eager/flash-like attention crossed with unquantized/NF4 storage.

Reproduce the calibration summary with:

python -m fitcheck.calibrate data/measurements/manifest.json --check --role calibration --role repeat

group                     runs    worst     mean
t4/eager/nf4              16      13.9%     3.2%
t4/eager/none             12       7.3%     2.6%
t4/flash/nf4              17       6.4%     2.6%
t4/flash/none             12       2.8%     0.9%

worst process-tier error with the shipped constants: 13.9%

For the 57 calibration/repeat rows:

The separate final holdout has 12 rows: tensor error is ±1.4%, process MAE is 5.4%, and the worst under-prediction is −15.3%. Boundary verdicts were 12/12 correct against the 14,000 MiB T4 safety budget. The accuracy holdout covers one Tesla T4 (sm_75), NF4, FP16 compute, LoRA r=16 on [q,k,v,o] (without double quantization), batch size 1, AdamW with FP32 states, gradient checkpointing, and sequence length 1024. It has six eager rows and six rows using SDPA's memory-efficient backend as a flash-like path. The six models are Qwen/Qwen2.5-7B-Instruct, Qwen/Qwen2.5-Coder-1.5B-Instruct, deepseek-ai/deepseek-coder-1.3b-instruct, meta-llama/Llama-3.2-1B-Instruct, meta-llama/Llama-3.2-3B-Instruct, and mistralai/Mistral-7B-Instruct-v0.3. This is not validation on a second GPU or real FlashAttention-2.

Honest limits

• Hardware evidence is limited to one Tesla T4 (sm_75). BF16 and real FlashAttention-2 are not validated; the flash-like path uses SDPA's memory-efficient backend on that card.
• The fitted C_overhead profiles cover only T4 × {eager, flash-like} × {none, NF4}. Other GPUs, INT8, checkpointing-off training, and inference use the conservative default profile: 500 MiB plus 5%.
• INT8 activation behavior has only one measured model, and serving decode transients are not modelled. Treat INT8 and high-concurrency inference estimates as lower-confidence results.
• The formulas target dense decoder-only models with a gated MLP. MoE, multimodal, encoder-decoder, sliding-window, compiled, and multi-GPU configurations are refused or outside the supported model.
• A cached config.json is enough to estimate offline, but an uncached model still needs Hugging Face access. FitCheck does not validate your exact training stack or prove that a run will not OOM.

Go deeper

• Technical specification — formulas, CLI contracts, validation rules, and limitations.
• Measurement archive — row format, roles, and reproducibility.
• Contributing — how to add measurements and regenerate fitted profiles.

The repository currently has 838 offline tests in the standard suite, 7 tests marked network, and 100% line coverage on all seven memory/ modules. An additional 13 measurement-harness tests run when torch is installed.

License

MIT. See LICENSE.