fitscout 0.1.1

Preflight training advisor: detect your hardware, size an LLM fine-tune to fit it, and emit a runnable config - before you download the model.

fitscout

A preflight training advisor for LLM fine-tuning.

fitscout runs before you fine-tune. It detects your hardware and environment (Colab / AWS / Azure / local), computes the numbers needed to make a given model fit - full vs LoRA vs QLoRA, micro batch and gradient accumulation, fp16 vs bf16, gradient checkpointing, 8-bit optimizer - and emits a runnable config. All of this before the model is downloaded.

fitscout does not train your model. It is an advisor that produces input for tools like the Hugging Face Trainer, Axolotl and DeepSpeed.

Why

• Accurate. Memory is estimated component by component (weights, gradients, optimizer, activations including the output cross-entropy, overhead) with principled formulas, not a fitted constant. Optional calibration measures a real step once per GPU and tightens the estimate into a guarantee.
• Generic by design. fitscout never branches on a model's name or family. It reads only architectural fields (layers, hidden size, head counts, KV heads, intermediate size, MoE experts). A brand-new model needs a new config, never new code - enforced by a test that fails if a model name appears in the source.
• Honest. Every estimate carries a confidence level and an uncertainty margin, and the default recommendation is conservative: it decides on the upper end of the estimate and never knowingly returns a HIGH-OOM-risk plan.

Install

pip install fitscout              # core advisor (dependency-light)
pip install "fitscout[all]"       # + introspection, GPU detection, torch, quant

The core has no required dependencies and runs anywhere. Heavy pieces are optional extras: introspect (transformers/accelerate/safetensors), hardware (pynvml), torch, quant (bitsandbytes).

Quickstart

The whole public API is three functions. The recommended import alias is fsc (fs is avoided as it is the import name of the PyFilesystem package). The flow is: learn what hardware and which model, decide how to train, then turn that into a config you can run.

inspect_environment() --.
                         |--> recommend() --> TrainingPlan --> export_config() --> runnable config
inspect_model(model) ---'

1. Import

import fitscout as fsc

2. Detect the hardware -> returns an Environment

env = fsc.inspect_environment()
# Environment(where='colab', gpu_name='NVIDIA T4', vram_gb=16.0,
#             bf16=False, compute_capability=7.5, num_gpus=1)

3. Profile the model (no weights downloaded, no model-name logic) -> ModelProfile

model = fsc.inspect_model("Qwen/Qwen2.5-7B")
# ModelProfile(params=7615283200, num_layers=28, hidden_size=3584,
#              num_attention_heads=28, num_kv_heads=4, intermediate_size=18944,
#              vocab_size=152064, tie_word_embeddings=False, num_experts=1, dtype_bytes=2)

4. Get a conservative plan -> TrainingPlan

plan = fsc.recommend(model, env, seq_len=1024)
# fits=True  method=qlora  precision=fp16
# micro_batch=2  grad_accum=8  grad_ckpt=True  opt_8bit=False
# oom_risk=MEDIUM   estimate=11.9 GiB (uncalibrated, +/-20%)

In words: this 7B model fits a 16 GiB T4 with QLoRA + gradient checkpointing at micro batch 2; estimated peak 11.9 GiB, medium risk.

5. Export a runnable config -> dict (or a ready-to-run script)

from fitscout.export import export_config
from fitscout.export.huggingface import render_script

cfg = export_config(plan, "Qwen/Qwen2.5-7B")   # TrainingArguments + LoRA + 4-bit
print(render_script(plan, "Qwen/Qwen2.5-7B"))   # a copy-paste training script

That is the whole loop: three public functions produce a plan, and one export call turns it into something you can run. Export backends: huggingface, axolotl, deepspeed.

For a readable summary in a notebook or terminal, print the plan as tables:

print(fsc.format_plan(plan, env))

+-------------+---------------------------------+
| fitscout - training plan                      |
+-------------+---------------------------------+
| GPU         | Tesla T4 (16 GiB) [colab]       |
| Fits        | yes                             |
| Method      | qlora                           |
| Risk        | MEDIUM                          |
| Estimate    | 11.9 GiB (+/-20%, uncalibrated) |
+-------------+---------------------------------+

Command line

fitscout Qwen/Qwen2.5-7B --vram 16 --seq-len 1024 --backend huggingface

GPU: CLI-specified (16 GiB)  where=cli
fits: True   risk: MEDIUM
method: qlora   precision: fp16
micro_batch: 2   grad_accum: 8   grad_ckpt: True   opt_8bit: False
seq_len: 1024
estimate: 11.9 GiB  (+/-20%, uncalibrated)

Omit --vram to auto-detect the GPU. Add --calibrate to use a cached correction factor.

Same model, different GPUs

A 7B model, seq_len=1024 - fitscout adapts the plan to the card:

GPU	VRAM	Plan
H100 / A100-80	80 GiB	LoRA, micro batch 4
A100	40 GiB	LoRA, micro batch 1
RTX 4090 / L4	24 GiB	LoRA + gradient checkpointing
T4 (free Colab)	16 GiB	QLoRA + gradient checkpointing
none	-	suggests the cheapest cloud GPU that fits

Calibration: estimate -> guarantee

The formula is conservative but uncalibrated estimates carry a +/-20% margin. To tighten it, measure one real step on your GPU:

from fitscout.memory.calibrate import calibrate

calibrate(model, "Qwen/Qwen2.5-7B")          # runs a tiny dry-run, caches the factor
plan = fsc.recommend(model, env, calibrate=True)   # now "calibrated", +/-10%

The correction is cached per (gpu, framework), so it carries over to other models on the same hardware.

API reference

Top level - fitscout

Function	What it does
inspect_environment()	Detect the runtime (Colab/AWS/Azure/local) and primary GPU (name, VRAM, bf16 support). Returns Environment.
inspect_model(source, *, dtype_bytes=2, trust_remote_code=False)	Profile a model's architecture without downloading weights (meta-device -> safetensors index -> analytical). Returns ModelProfile.
recommend(model, env=None, seq_len=2048, calibrate=False)	Pick a conservative full/LoRA/QLoRA plan that fits the hardware; never returns HIGH risk. Returns TrainingPlan.
format_plan(plan, env=None)	Render the plan and its memory breakdown as ASCII tables (same output as the CLI). Returns str.

Data contracts: Environment, ModelProfile, MemoryEstimate (.upper_gb gives the conservative upper bound), TrainingPlan.

Memory - fitscout.memory

Function	What it does
estimator.estimate_memory(profile, *, method, micro_batch, seq_len, grad_ckpt, opt_8bit, ...)	Combine all components into a MemoryEstimate with confidence + margin.
components.*_gb(...)	The individual terms: weights_gb, gradients_gb, optimizer_gb, block_activations_gb, output_activations_gb, dequant_buffer_gb, overhead_gb.
calibrate.calibrate(profile, model_ref, ...)	Run one real step, derive measured/estimated, cache it per (gpu, framework).
calibrate.correction_factor / measure_peak_gb / CalibrationCache	The calibration building blocks.

Model - fitscout.model.introspect

Function	What it does
inspect_model(...)	Same as the top-level function.
extract_arch_fields(config)	Read architectural fields from a config dict via candidate key names.
analytical_param_count(fields)	Estimate parameters from the config (last-resort fallback).

Environment - fitscout.environment

Function	What it does
detect.detect_where(environ=None)	Return "colab" / "aws" / "azure" / "local".
hardware.inspect_environment()	Read the GPU via torch, then pynvml, else report no GPU.

Export - fitscout.export

Function	What it does
export_config(plan, model_ref, backend="huggingface")	Turn a plan into a backend config dict (huggingface / axolotl / deepspeed).
available_exporters() / get_exporter(name) / register(exporter)	Inspect and extend the exporter registry.
huggingface.render_script(plan, model_ref)	Emit a runnable training script.
axolotl.render_yaml(plan, model_ref)	Emit an Axolotl YAML config.

Cloud - fitscout.cloud.catalog

Function	What it does
cheapest_fit(required_gb)	Cheapest single catalog GPU that holds the requirement.
rightsize(required_gb, allow_multi=True)	Cheapest single- or multi-GPU suggestion.

Accuracy and honesty

fitscout is validated against real measured peak VRAM - see validation/. On a Colab T4 (QLoRA, 8-bit optimizer, gradient checkpointing, seq_len 1024):

Model	Estimate (GiB)	Measured (GiB)	Error
Qwen2.5-1.5B	5.0	4.0	+25%
Qwen2.5-3B	6.0	5.1	+18%
Qwen2.5-7B	9.7	10.0	-3%

Estimates lean conservative (the positive direction is safe). The per-GPU correction factor is largely model-independent but not perfectly so - on this data it spans ~0.80-1.03 across model sizes - which is why a calibrated estimate keeps a (narrower) margin rather than claiming an exact number.

License

MIT.