Loggetta

Plan the run. Train the model. Keep the evidence.

Loggetta checks your hardware, chooses a supported configuration for a Mixture-of-Experts (MoE) model, and runs QLoRA fine-tuning. It saves the plan and a JSON run report with memory use, timing, and checks that the selected optimizations actually ran. One install includes the runtime (experts4bit-qlora) and the GPU kernels (grouped-nf4-gemm).

pip install loggetta
loggetta inspect
loggetta plan Qwen/Qwen3-30B-A3B --seq 2048

The plan shows where weights will live, estimated memory use, and why alternatives were rejected. It checks the budget before downloading model weights. Estimates can miss; they are not an out-of-memory guarantee.

New in 0.4.0

• Dense models: planned, not supported for training. Dense plans are estimates checked in sample; out of sample (DQ7) they missed, and the DQ10 reading is pending. Dense execution needs --allow-development-executor until DQ8's 24 GB reading passes. See Dense training.
• The DQ10 reserve policy is opt-in, for registered dense runs. Defaults do not change.
• MoE training estimates price the grouped_nf4 backward pass. OLMoE plans on an RTX A2000 were about 0.2 GiB short; in sample they now cover the measured peak. Near a budget, a plan may pick the reference kernel or host residency where it picked resident grouped_nf4 before.
• Single-stream serve plans name the speed-ups a default server runs for the model's family, and quote a decode speed only from a measured run of the same setup.
• Fix: resident grouped_nf4 training runs again with experts4bit-qlora 0.49.0 or later. Loggetta 0.3.x stopped before the first step (#47); the workaround was E4B_ABSMAX_DQ=0.

Full list: CHANGELOG.

Train on your data

loggetta train Qwen/Qwen3-30B-A3B \
  --dataset ./data/train.jsonl --format text \
  --seq 512 --micro-batch 1 --steps 20 --seed 42 \
  --out runs/my-training --adapter-out adapters/my-adapter

Local JSONL, JSON, CSV, Parquet and TXT files, Hub datasets, Alpaca instructions and text-only chats are supported. Data is validated and tokenized before weights load. Chat data trains only the assistant turns by default. Reload the adapter with loggetta.load_adapter("adapters/my-adapter"). See the training guide.

To run a saved decision later: loggetta plan MODEL --out plan.json, then loggetta execute plan.json --out runs/. Pass earlier run reports back with --observations runs/ and later plans use those measurements.

Measured results

The included runtime and kernels do the compute; these are matched training runs, not planner benchmarks.

The comparison used torch 2.12.1+cu130 and transformers 5.5.0 for both frameworks, with matched adapters, initialization and tokens. Loggetta does not predict throughput.

Models

Every row is supported by the included runtime's QLoRA path. Hybrid models (Qwen3.6, Granite-4.0-H, LFM2, Nemotron-H) need --packing concat with chat or Alpaca data. The runtime's capability register is the authority.

Scope

Single-GPU MoE planning and QLoRA training; serving placement is planned, not launched. Not yet: multi-GPU, throughput prediction. GPU runs need Linux, an NVIDIA CUDA GPU and a compatible PyTorch. Pre-1.0.

Dense models are not supported for training yet: their plans are estimates checked in sample (DQ7 missed out of sample, DQ10 pending), and their training runs only behind --allow-development-executor until DQ8's 24 GB reading.

GitHub · Results · Architecture · Research and releases