pichak

Fine-tuning hyperparameters derived from your machine, with the measurement printed next to every number.

pip install pichak

from pichak import derive

plan = derive(
    data="train.jsonl",
    tokenizer=tok,
    step_fn=lambda b: one_real_forward_and_backward(b),
    named_weights=model.named_parameters(),
)
print(plan.report())

  seq_len            512
                     token p95 416 over 512 real rows, rounded up to 512; p99 is
                     657 and max 943, so 5% of rows lose their tail at this length

  micro_batch        3
                     a doubling ramp with a real forward+backward at each rung:
                     1=ok, 2=ok, 4=ok, 8=SPILL. 8 SPILLED to system memory — it did
                     not raise, it got 1.6x slower per sample, which is the Windows
                     failure mode, so 4 is the largest that fit, peaking at 5.05GB
                     of 5.05GB; kept 3 to leave 12% for optimizer state

  learning_rate      0.000282
                     1/32 of the scale at which one Adam step would rewrite rather
                     than perturb. ||W||_F/sqrt(numel) over 26 matrices; the
                     smallest binds and it is b.1.f.2.weight at 0.00902

  NOT MEASURED (1): target_tokens=65536

Why not just use a good default

Because you cannot tell a good default from a bad one when it fails.

micro_batch=4 tells you nothing about whether 4 came from a measurement on your card, a heuristic in a blog post, or a number someone picked in 2023 for a different model. So when it OOMs at step 40 you bisect instead of reading.

Every number pichak returns carries the sentence that produced it. Anything it could not measure is collected under plan.constants() and printed together, so a constant can never quietly pass for a derivation.

plan.micro_batch          # 3
plan.why("micro_batch")   # the ramp, rung by rung, and which one failed
plan.constants()          # {'target_tokens': 65536}

It catches the failure that does not raise

On Windows, a batch that exceeds VRAM does not throw. The driver pages the excess to system memory and the step simply crawls. Measured on a GTX 1060 6GB with a 6-layer model at sequence 512:

Both "work". The first is 23x slower and nothing in the logs says why. On a healthy ramp, seconds-per-sample is flat or improves as the batch grows — bigger batches amortise fixed costs. A sudden jump means the batch did not really fit.

What it derives, and from what

Everything is optional. Pass only a corpus and a tokenizer and you get a sequence length; pass a step_fn and you get the batch. It will not invent a number it could not measure — a plan with three derived values and an honest gap is more useful than one with ten you cannot tell apart.

The learning rate, since it is the surprising one

An Adam update moves a parameter by roughly lr, whatever the gradient's scale. So there is a learning rate at which one step rewrites a weight matrix instead of nudging it: where lr * sqrt(numel(W)) reaches ||W||_F. Every sane learning rate is a fraction of that, and the fraction is what schedules argue about.

Measured, at the ranks behind this library:

rewrite/2    2.26e-3   destroyed the model twice (CE 1.74 -> 10.4, 1.11 -> 19.3)
rewrite/32   1.41e-4   the first scale that actually learned
rewrite/45             where a hand-tuned LoRA run at 2e-4 landed

pichak opens at rewrite/32 — the top of the range that works. Opening hot costs the model; opening cold costs time. With a penalty that asymmetric you start at the bottom of what is known.

This is not tuned for your dataset. It is the scale at which updates are the right size for these weights. It replaces "I copied 2e-4 from a post about a different model", which is the actual alternative.

Measurement tools, usable on their own

pichak gpu                    # virtualisation, launch latency, transfer bandwidth
pichak disk "models/*.safetensors"   # queue-depth sweep, page cache bypassed
pichak corpus train.jsonl mistralai/Mistral-Small-24B-Instruct-2501

from pichak.measure import disk_queue_depth, transfer_bandwidth, virtualisation

These exist because the numbers people quote are rarely the numbers their machine gives:

• A buffered disk benchmark reported 4079 MB/s on a drive rated 2100 — that was the page cache. Unbuffered, the same drive peaks at queue depth 3 and gets slower past it.
• A rented A6000 measured D2H 457 MB/s against H2D 1502, and pinned memory — the standard fix — bought nothing. Nothing about PCIe explains that; it was a fabric, and it decided where hidden states could live.
• One rented machine had 84.8us kernel launches against a normal 2-5. A workload issuing 400,000 launches per step would have spent nine hours on overhead and looked slower than a 2016 card.

Install

pip install pichak            # the plan, the disk sweep — no dependencies
pip install pichak[torch]     # the ramp, the learning rate, the GPU measurements
pip install pichak[hf]        # + transformers, for the corpus CLI

Python 3.9+. The core has no dependencies at all; torch is only needed for the parts that touch a GPU.

Where the numbers come from

Every measurement quoted here was made while fine-tuning a 24B model on a 6GB GTX 1060 and on a rented A6000 held down to the same 6GB. The raw logs — including the six runs that failed before one worked — are at flap-findings, along with the five hypotheses that sounded right and were wrong.

Licence

MIT. By Oleksandr Pichak.