Loggetta

One install for planning and running single-GPU MoE QLoRA workloads.

pip install loggetta

That installs Loggetta plus its current runtime and kernel stack:

• Loggetta — hardware inventory, planning, ExecutionPlan, execution orchestration, and ExecutionReceipt
• experts4bit-qlora — model loading, QLoRA, training, serving mechanisms, and CPU/NVMe offload
• grouped-nf4-gemm — packed low-bit kernels and residency primitives

Loggetta examines the model, machine, workload, and constraints before loading model weights. It chooses a supported setup, explains why alternatives lost, and refuses configurations estimated not to fit. For supported training plans it dispatches into the included runtime and records what actually happened.

loggetta inspect
loggetta plan Qwen/Qwen3-30B-A3B --seq 2048
loggetta train allenai/OLMoE-1B-7B-0924 --seq 512 --micro-batch 2 --steps 12 --out receipts/

Without --dataset, train runs a short demonstration on tatsu-lab/alpaca (full-sequence loss) with the default warmup + cosine learning-rate schedule. To train on your own data and keep a reusable adapter:

loggetta train Qwen/Qwen3-30B-A3B --dataset ./data/train.jsonl --format chat \
  --seq 1024 --micro-batch 1 --epochs 1 --out runs/my-run --adapter-out adapters/my-adapter

The dataset is validated, tokenized and profiled before any weights load, and the plan carries that profile. For chat and Alpaca data the defaults are assistant-only loss (--loss), isolated packing so each example attends only to itself (--packing), and warmup + cosine decay (--lr-schedule). Reload the adapter with loggetta.load_adapter("adapters/my-adapter"). See the training guide.

A feasible plan is an estimate, not an OOM guarantee. Serving placement can be planned; Loggetta does not launch the server yet.

Plan, run, measure

A plan is a serializable artifact, not just terminal output. It records the selected backend setup, budgets, memory estimates, rejected alternatives, warnings, and anything the planner does not model. You can save it, inspect it, then execute that exact decision:

loggetta plan allenai/OLMoE-1B-7B-0924 \
  --seq 512 --micro-batch 2 --steps 12 --out plan.json

loggetta execute plan.json --out receipts/

The resulting receipt records the plan plus measured allocator/reserved/driver memory, timing, correctness checks, and runtime provenance. Earlier receipts can be passed back as observations so later plans use measurements from the same model/setup when available:

loggetta plan allenai/OLMoE-1B-7B-0924 \
  --seq 512 --micro-batch 2 --observations receipts/

The planner does not load weights while choosing. With --dataset, every row is validated and tokenized first; then the model topology and the hardware are read. Candidate selection is deterministic policy over those facts, the data profile and the constraints.

Measured speed

These are measurements of the runtime/kernel stack that Loggetta installs and dispatches into, not speedups caused by the planner.

Evidence: Unsloth same-stack · second-host replication · Axolotl/Unsloth matched work · H100 pipeline

Models: supported vs tested

Runtime-supported means the included experts4bit-qlora fast-training path is evidence-gated supported with a real-weight PASS receipt. The Loggetta column says what this repository itself has exercised.

Packing on hybrid models. With chat or Alpaca data the default isolated packing is refused for a model whose layers mix tokens through a state (Qwen3.6-35B-A3B, Granite-4.0-H-tiny, LFM2-8B-A1B and, by its structure, Nemotron-H) or whose attention is not described (DeepSeek-V2-Lite): resetting positions cannot isolate examples there. Pass --packing concat for those models.

Also exercised by the planner but not advertised as supported expert-QLoRA rows: DeepSeek-V2-Lite is planned with attention LoRA disabled because its MLA attention is not described by the current adapter path; gpt-oss-20b is deliberately refused for expert QLoRA because its biased/clamped expert structure does not satisfy ExpertsLoRA's contract.

Backend support is evidence-gated and can move independently of Loggetta's own test matrix. See the runtime capability register and Loggetta's measured results.

Memory planning

Loggetta labels each estimate as measured, derived, inferred, or heuristic instead of presenting every number as equally certain. An ExecutionReceipt records estimated vs measured memory and can be fed back into later plans.

Selected checks:

• A2000 training runs across OLMoE and Granite families put allocator estimates within 0.01-0.21 GiB of measured peaks.
• Qwen3-30B-A3B on an RTX 5090: allocator 22.09 GiB estimated vs 21.91 GiB measured; after receipt-calibrated runtime overheads, process peak 24.54 GiB planned vs 24.34 GiB measured.
• OLMoE serving on an A2000: planned VRAM/DRAM/NVMe expert placement matched the server's 272 / 421 / 331 split; allocator 2.052 GiB planned vs 2.051 GiB measured.

Loggetta does not currently predict throughput. It may use measured evidence to order valid setups, but unknown speed remains unknown.

Current scope

Available: one-command install of the full stack; single-GPU MoE planning; saved ExecutionPlan and ExecutionReceipt; QLoRA training on your own data (local JSONL/JSON/CSV/Parquet/TXT or Hub datasets; text, Alpaca or chat) with reusable adapters; QLoRA training execution; device/host/storage serving placement planning; measured-memory feedback; explicit refusals.

Not claimed: first-class dense-model planning; multi-GPU planning/execution; calibrated throughput prediction; a Loggetta server-launch command; universal exposure of every mechanism in the lower packages.

GPU execution currently targets Linux + NVIDIA CUDA and requires a compatible driver/PyTorch environment. The package is pre-1.0.

More detail

The PyPI page is intentionally short. The GitHub repository carries the architecture, full benchmark caveats, receipts, family sweeps, and reproducibility notes:

• Full README
• Results
• Architecture
• experts4bit-qlora capabilities