litetune

Fine-tune a small model, convert it for LiteRT-LM on native devices, and compare the result with the model it came from.

Getting from a Hugging Face checkpoint to a model that works inside your app is a long road — LoRA, merge, export to .litertlm, bundle metadata — and a mistake at any step produces a file of the right size that loads without error and is broken while every check stays green. litetune walks that road and knows the traps on it.

The output is a .litertlm bundle: what LiteRT-LM loads — natively on Android, iOS, macOS, Linux and Windows, with GPU acceleration on each. The flutter_gemma plugin runs it through the LiteRT-LM C API on all five; Google's AI Edge Gallery loads the file directly if you only want to try it on a device. Web exists as a text-only preview that supports neither function calling nor LoRA, so a tuned tool-calling model is native-only for now. The same model also runs on a Snapdragon 8 Elite NPU — hand-compiled through litert-torch's npu_export stages at the cache_length this project measured while isolating LiteRT-LM#3508 — and, for single-chunk prompts, on an Intel Lunar Lake NPU. The recipe, the numbers and the conditions are under Limitations; the compile is not in convert.

Any task shaped as prompt → completion works. What "correct" means is the one thing you choose: --scorer tool-call for function calling, where the operation name and every argument value must match, or --scorer exact-text where there is one right string. Everything after scoring — the paired comparison, the intervals, whether a difference resolves, the exit code — reads only whether each example was right, so it does not know or care which task you brought.

• prepare — validate the data and make a reproducible train/held-out split
• tune — LoRA or full fine-tuning, with the wiring the export needs
• convert — checkpoint to .litertlm, across quantization recipes
• verify — compare the converted model with its float reference before you ship
• bundle — package the artifact with what was measured about it

litetune env shows the environments the stages cached, and --clean removes them.

tune and verify's float reference resolve the device once per run, by asking the stage environment's own torch, in this order: CUDA if torch.cuda.is_available(), then Apple's GPU through Metal (mps) if torch.backends.mps.is_available(), then the CPU. When a Mac lands on the CPU the report carries what torch answered for is_built() and is_available() and names no cause beyond them, since is_available() gives none. Every run records the operating system, its version — macOS's product version on a Mac, platform.release() elsewhere, which on Linux is the kernel's — the machine and, on macOS only, the chip. convert currently uses litert-torch's CPU export path; the upstream command exposes no host-device flag, and litetune hides CUDA and HIP devices so that path also works on a runner with no GPU. Set LITETUNE_DEVICE=cpu in the shell you launch from to put tune and the reference on the CPU whatever the machine has. Unset, or auto, is the order above; any other value is refused before anything runs, and when the device came from the variable the report says so among its limitations, beside what the probe found. The converted model's backend is chosen separately, by verify --backend (cpu unless you say gpu), and LITETUNE_DEVICE does not reach it. CPU alone is workable at 270M and the first thing you will want to change above about 1B.

Before a training or reference run starts on mps, litetune sets torch's MPS allocator limits from the memory the Mac has available at that moment. The budget is the smaller of Metal's recommended working set (torch.mps.recommended_max_memory()) and the available memory less 3 GiB, where available is the free and the pageable external page counts (vm.page_free_count, vm.page_pageable_external_count) times hw.pagesize — litetune's own conservative measure, built from two of the counters XNU's jetsam path sums (VM_CHECK_MEMORYSTATUS in osfmk/vm/vm_page.h), not a figure macOS reports; it counts neither other apps' memory nor pages that would have to go to swap; PYTORCH_MPS_HIGH_WATERMARK_RATIO is the budget over the working set, PYTORCH_MPS_LOW_WATERMARK_RATIO is 0.8 of that, and PYTORCH_ENABLE_MPS_FALLBACK=0 — torch's own default, written out — makes an operation MPS does not implement raise instead of running on the CPU. A value you set for any of the three is kept and recorded as yours, after a check: the ratios must be plain decimals from 0 to 2.0, the low one no higher than the high one, and the fallback a plain integer; anything else is refused before the run starts. A high ratio you set is the limit, so the 1 GiB floor below does not apply to it; the report records litetune's computed budget beside the limit in force, and says so when your 0.0 removed the limit or your non-zero fallback let operations run on the CPU. A budget litetune computes under 1 GiB is refused rather than started, and so is a Mac whose kern.memorystatus_vm_pressure_level is not 1 (normal): quit what is holding memory, or set LITETUNE_DEVICE=cpu. On mps, tune trains in float32 unless you pass --dtype, pads each batch to a multiple of 32 tokens (never past --max-seq-length), and defaults to one example per forward pass while accumulating the gradients for the --batch-size optimiser batch. CPU and CUDA keep the previous default of one forward per optimiser batch. Pass --micro-batch-size to choose another split on any device; it cannot exceed --batch-size. The report records both sizes and the number of accumulation steps. litetune also records the largest MPS memory counters it sampled. An MPS out-of-memory error is reported as the machine's, not the recipe's, and so are an operation MPS does not implement, bfloat16 refused before macOS 14 and a watermark ratio torch rejected — each with LITETUNE_DEVICE=cpu as the way forward. PR #63 records one guarded Qwen2.5-0.5B-Instruct training run on MPS beside a separate CPU baseline with the same 2,400 examples and 300 optimiser steps. That is not a general speed claim or a model-quality result: no score, CUDA comparison or completed MPS verify run has been measured here. Bring your own checkpoint and skip the first two steps, or bring a .litertlm and its float checkpoint and run only verify.

Alpha. Measured end to end on six models: google/functiongemma-270m-it with the tool-call scorer, and google/gemma-3-270m-it, google/gemma-3-1b-it, Qwen/Qwen3-0.6B, Qwen/Qwen2.5-0.5B-Instruct and google/gemma-4-E2B-it with exact-text on the same 77-way intent task — every headline candidate evaluated on LiteRT-LM's CPU backend, and two Qwen3 artifacts also evaluated on a phone's CPU and GPU, all in MEASUREMENTS.md. Qwen3.5 exports and needs no flags from litetune, only a transformers floor, and has no quality number. Gemma 4 exports once you name the variant — E2B or E4B — because the chat template override is per-variant; a bare gemma-4 is refused rather than guessed at, and the refusal names the flag to pass if you want to choose the template yourself. It carries two numbers the others do not: an earlier conversion cost measured on its untouched base weights, which used prompts listing all 77 labels and so does not sit beside the five above it, and a comparison of two LoRA runs that differ only in which projections they adapt — the seven litetune names against the two peft would have chosen, which is 24 points apart. On a multimodal checkpoint litetune keeps a LoRA run inside the text tower; tune says so in its own output. Try it on yours and open an issue.

Install

pip install litetune

Or through Homebrew, which brings its own Python 3.12:

brew install DenisovAV/tap/litetune

Linux, macOS or Windows, Python 3.10–3.12 — with convert needing Linux x86_64 or an Apple Silicon Mac, for the reason below. On Linux you also need libvulkan1 — litert-lm dlopen()s a Vulkan-linked library even for the CPU backend, and without it every invocation, --help included, dies in under a second:

sudo apt-get install -y libvulkan1     # Debian/Ubuntu

macOS needs nothing extra; Colab works out of the box.

Windows runs everything but convert. prepare, tune, verify and bundle install and run on Windows x64: every pin in those environments has a wheel that installs there — win_amd64 for the ones with native code, litert-lm-api included, and py3-none-any for the rest. convert cannot, because litert-converter — the MLIR converter litert-torch depends on — has never published a Windows wheel or a source distribution, so the install fails while pip is still resolving. Upstream tracks it as litert-torch#968, where a collaborator says Windows is planned and gives no date. Google's verified platforms page says the same: "Windows: litert compile and litert convert not supported yet." The same gap covers ARM Linux and Intel macOS. There are wheels for macOS on Apple Silicon, though upstream's README still names Linux as its operating system, and this project has only ever converted on Linux.

Two ways round it. Run everything under WSL2 with Ubuntu — install pythonX.Y-dev, or the build stops on a missing libpython3.x.so (litert-torch#74), and keep the checkout and ~/.cache/litetune on the Linux filesystem rather than under /mnt/c. Or convert on any Linux machine and bring the .litertlm back: verify takes it with the float checkpoint it came from and asks nothing of the export environment.

tune on Windows trains on the CPU unless you say otherwise, and that is PyTorch's packaging rather than litetune's: every CUDA dependency of torch==2.5.1 is marked platform_system == "Linux", so the wheel PyPI serves Windows is the CPU build. PIP_EXTRA_INDEX_URL=https://download.pytorch.org/whl/cu124 reaches the stage install and gets you the CUDA one — after litetune env --clean, because an environment's identity is a hash of the interpreter and the pins, which an index does not change, so a cached CPU environment would be reused in silence. Either way the run records what it got: device and cuda_build are in the training metrics.

Python 3.13 runs tune, prepare and bundle but not convert or verify: each stage builds its own environment from the interpreter you launched, and numpy==2.0.2 — pinned by the export toolchain — stops publishing wheels after 3.12. Past a ceiling the command refuses and names the pin that set it.

Each of tune, convert and verify builds its environment on first use and caches it. Measured on macOS:

On Linux the training environment is larger: the torch wheel pulls its CUDA dependencies, several hundred megabytes each.

litetune env shows what is on disk and litetune env --clean removes it; the next stage that needs one rebuilds it. Worth knowing because a provision that died halfway leaves a directory that looks like a working one from the outside, and because the cache key includes the interpreter — running litetune under two Pythons builds two sets.

Which runtime each version measures on

A number from verify belongs to the runtime and the CPU that produced it -- its model name and its instruction-set features together. On one CPU model litert-lm 0.17.1 and 0.16.1 gave the same per-token scores on 1 of 36 prompt-bundle scorings, and one runtime on an Intel and an AMD server CPU on none, while each runtime on one CPU model reproduced every score to the bit -- 0.17.1 on the Intel model with 1, 4, 8, 16 or 32 threads alike (MEASUREMENTS.md). So each litetune release pins one runtime, and from 0.2.0 every verify manifest records the litert-lm version and the CPU its candidate ran on where the driver could read them, and says so when it could not read the version; 0.1.x recorded the version on the tool path only and the CPU nowhere.

To reproduce a published number, install the release that took it: every section of MEASUREMENTS.md that verify produced on litert-lm 0.16.1 was taken with 0.1.x, so pip install litetune==0.1.9. The ones taken on 0.17.1 ran main before 0.2.0, at a commit each names, on the runtime 0.2.0 pins, and say whether their bundle was converted by 0.1.x or by that commit. The section taken on 0.18.0 ran the code of pull request #67 before it was merged, on the runtime 0.4.0 pins, and says which bundles it converted again. The phone sections ran on the device. Up to 0.1.9 the conversion environment pinned litert-torch-nightly but not the quantizer and converter packages it requires, so it took whichever were newest the day it was built; convert recorded what it got in its report's toolchain.resolved.

Your data

One JSON object per line. Scoring rows need a prompt and a target; training rows add the completion text the model should produce, or let prepare derive it from the target.

The shape of the target is how you declare the task. An object with a name is a tool call:

{"prompt": "<start_of_turn>developer\nYou are a model that can do function calling with the following functions\n<start_function_declaration>declaration:set_alarm{description:<escape>Sets an alarm<escape>,parameters:{properties:{hour:{description:<escape>Hour of the alarm<escape>,type:<escape>STRING<escape>}},required:[<escape>hour<escape>],type:<escape>OBJECT<escape>}}<end_function_declaration>\n<end_of_turn>\n<start_of_turn>user\nset an alarm for 7\n<end_of_turn>\n<start_of_turn>model\n", "target": {"name": "set_alarm", "args": {"hour": "7"}}}

A bare string is the answer itself:

{"prompt": "classify the sentiment: it was fine", "target": "neutral"}

Two shapes rather than a target plus a --target-kind, because those two could disagree and a shape cannot disagree with itself. Match it with --scorer when you get to verify.

The prompt is exactly what your application will send the model. An application that renders the tool declarations and every turn marker into the prompt itself sends a prompt the runtime must not template again, and tune trains it prerendered. The sentiment row is bare text for a runtime that applies the model's own chat template, so it trains runtime_rendered — as does a tool call whose declarations the runtime renders, below in Tool calling through the runtime. tune tells the two apart by the control tokens in the prompts, and refuses a file that mixes them unless you declare which one it is.

prepare splits one raw file into train.jsonl and heldout.jsonl; it refuses malformed JSON and rows with no prompt. Given --tokenizer it also reports the token-length distribution, so a row too long for the sequence limit fails before you rent a GPU rather than after.

The held-out rows — a fifth of the file by default, --heldout-fraction or --heldout-size to change it — are never trained on. Scoring a model on rows it was fitted to measures memorisation rather than whether it answers new inputs. The split is derived from the file's content hash, so re-running prepare puts the same rows on the same side.

From your data to a shippable bundle

Five commands, in order. Each is separate because each fails differently, and a single run would hide which one you are in.

# 1. Validate and split the data. Seconds.
#    Without --tokenizer it cannot measure token lengths, so it splits the file
#    and exits 4 — "could not check" — rather than implying the rows all fit.
litetune prepare --data raw.jsonl --output-dir data --context-length 1024 \
                 --tokenizer google/functiongemma-270m-it

# 2. Fine-tune. Chooses CUDA, then MPS on a Mac, then CPU. Declare the
#    prompt mode rather than leave it to be read off the prompts: these carry
#    FunctionGemma's control tokens, so prerendered. It is recorded beside the
#    checkpoint, where steps 4 and 5 take it from. On a CPU add --dtype float32:
#    bfloat16 runs single-threaded there (see the flag table below).
litetune tune --model google/functiongemma-270m-it --data data/train.jsonl \
              --output-dir tuned --method lora --prompt-mode prerendered

# 3. Convert, sweeping recipes rather than trusting a default.
litetune convert --model tuned/model --output-dir artifacts \
                 --recipe dynamic_wi8_afp32 --recipe weight_only_wi8_afp32

# 4. Compare the converted model with its float twin.
#    `convert` names the artifact; look the filename up rather than build it.
#    The prompt mode is read from the record step 2 left beside tuned/model,
#    and a --prompt-mode that disagrees with it is refused.
litetune verify --model artifacts/weight_only_wi8_afp32/<name>.litertlm \
                --reference tuned/model --data data/heldout.jsonl \
                --json > manifest.json

# 5. Package the artifact with what was measured about it.
litetune bundle --output-dir bundle \
                --model artifacts/weight_only_wi8_afp32/<name>.litertlm \
                --declarations tools.json \
                --base-model google/functiongemma-270m-it \
                --base-model-revision <commit-sha> \
                --adapter tuned/adapter \
                --train-metrics tuned/metrics.json \
                --verify-manifest manifest.json

Step 3 already gives you something shippable — one .litertlm per recipe, under artifacts/<recipe>/. Steps 1–3 are also the part most tooling makes you assemble by hand; see What it knows for what they do beyond calling the exporter yourself.

Steps 4 and 5 are what makes it trustworthy. --reference is the float twin: the same weights before conversion. That is what makes the difference between the two the conversion cost rather than a mixture of that and whatever training did. Point it at a different checkpoint — an untuned base, say — and pass --reference-role untuned_base, and both the training gain and the conversion cost come back unavailable, because one number cannot separate two effects.

If you already have a .litertlm and the checkpoint it came from, step 4 runs on its own.

Recipes

The recipes are AI Edge Quantizer's, applied by litert-torch export_hf during convert, and --recipe passes a name litetune does not know straight through to it. litetune defines exactly one of its own, shipped as a quantizer recipe file inside the package; what it adds to the rest is a measurement of what each costs on your task. The costs below were taken on litert-lm 0.16.1, the runtime 0.1.x pins, and the four-bit rows give 0.17.1's beside them, the runtime litetune 0.2.0 – 0.3.0 pin. The two runs differ in runtime, transport and possibly CPU; see MEASUREMENTS.md for both.

--recipe has no default. A sweep of one is not a comparison. At four bits, on litert-lm 0.16.1, every model measured here lost accuracy the sample resolves, and how much does not follow from its family or its parameter count: 3.50 points on Qwen3-0.6B, 7.67 on Qwen2.5-0.5B, 8.83 on Gemma 3 1B and 34.83 on the 270M — see MEASUREMENTS.md for the intervals, the refusals and what those numbers do not establish.

Other flags that decide something

Tool calling through the runtime

The walkthrough above renders the declarations into every prompt itself. The other way is to let the runtime do it: bare prompts, and the declarations passed as a file to create_conversation(tools=...). Then they are an input to every stage, not only to bundle:

litetune prepare --data raw.jsonl --output-dir data --context-length 1024 \
                 --tokenizer google/functiongemma-270m-it \
                 --base-model google/functiongemma-270m-it --declarations tools.json

litetune tune --model google/functiongemma-270m-it --data data/train.jsonl \
              --output-dir tuned --method lora \
              --prompt-mode runtime_rendered --declarations tools.json

# convert as in step 3, then:
litetune verify --model artifacts/<recipe>/<name>.litertlm \
                --reference tuned/model --data data/heldout.jsonl \
                --declarations tools.json --json > manifest.json

tools.json is a list of OpenAI function objects, the shape bundle takes.

The order of its keys is settled for you. The runtime prints a declaration's keys in the order it is given; FunctionGemma's own chat template sorts them with dictsort, which ignores case. So litetune sorts every mapping in the file the same way when it reads it, and the training prompt and the runtime then render the same tokens — the rendering check compares them on every verify.

It refuses what the two still render differently, and what a call could not carry back:

• any schema key the template does not print — among them nullable, default, format, minimum, additionalProperties, a function-level strict, and enum on anything but a string;
• a tool or property with no description;
• a property named description, type, properties, required or nullable, and two properties equal but for case;
• an empty collection, and an object property with no properties;
• a type written other than as one of the seven JSON Schema names, in lowercase or in capitals;
• a tool or argument name the runtime's call parser cannot read ([a-zA-Z_][a-zA-Z0-9_.-]*, and not call, true, false, null or an exponent like e5 or e-3, which its lexer reads as other tokens);
• a key given twice in one object, a tool declared twice, and text that is not valid Unicode;
• a description, an enum value or a property name holding <escape>, a declaration marker or a turn marker, which would end the declaration it is written in.

Some are capability you give up — nullable, the reserved names, enum on a number, and OpenAI's strict mode; the rest you fix by writing the file differently. Whatever you remove, remove from what your application sends too: the runtime renders what it is given. google/mobile-actions meets one of them itself — its tools with no arguments carry "properties": {}. The disagreement is Google's: LiteRT-LM#3638.

A call is trained the way the runtime reads one: inside <start_function_call> and <end_function_call>, strings between <escape> markers, numbers, booleans and null bare, and the arguments in the same order as the declarations — call:set_alarm{hour:7,label:<escape>wake<escape>}. Only what the runtime reads back is trained: a string holding an escape, the end-of-call marker or one of FunctionGemma's stop tokens (<end_of_turn>, <start_function_response>, <eos>), NaN or infinity, an integer a double cannot hold exactly and a name outside its grammar are refused with the row named, and so is a target whose arguments contradict its declaration. The order is not cosmetic: with constrained decoding on, the runtime enforces the declared property order, and an argument out of it is dropped from the call. So a runtime_rendered bundle ships its declarations in the order the model learned, and its contract's declarations_sha256 names that list.

Who serves it this way. LiteRT-LM's Python API, as create_conversation(tools=...) handed each entry of the bundle's declarations.json whole, {"type": "function", ...} — which is how verify asks. A Python function handed as a tool gets a schema the binding writes itself, and does not render what was trained. Constrained decoding is off unless you pass a ConstrainedDecodingConfig that enables it. Automatic tool calling is on unless you turn it off: with it on, the binding runs the tools itself and loops until the model answers in prose, so you are never handed the call; verify measures with it off.

On Kotlin, an OpenApiTool returning each entry's function object from the bundle's declarations.json — not the whole {"type": "function", ...} entry, which it refuses — registered in the file's order. Parse the file into a JSON object that keeps its keys' order and hand it on; a data class serialised back out can reorder the keys. The reflection-based @Tool path writes its own keys, order and nullable, so it renders what was trained only for a tool declared with no parameters at all. Constrained decoding there is ExperimentalFlags.enableConversationConstrainedDecoding, off by default and global to the process, read when a conversation is created. Automatic tool calling is on by default in Kotlin too; turn it off to be handed the call. Up to LiteRT-LM 0.16.1 the runtime reads every number in a call as a double, so an integer argument arrives as 7.0; from 0.17.1 an integer that fits in 64 bits arrives as one (fc_parser.rs). Read it as a number either way.

flutter_gemma uses this path for FunctionGemma on a .litertlm from core 1.8.4 with flutter_gemma_litertlm 1.7.1: it hands the runtime the declarations (_nativeToolsJson in lib/src/ffi/ffi_inference_model.dart) and creates the conversation with constrained decoding on (lib/src/ffi/litert_lm_client.dart), so the grammar-on number is its path; releases before core 1.8.4 render FunctionGemma's declarations in Dart and are not served the way it is measured. flutter_gemma_litertlm 1.8.0 ships LiteRT-LM 0.17.1; flutter_edge_ai_litertlm 1.10.0, its renamed successor, ships 0.18.0, the runtime this litetune measures on.

verify picks the path from the model, not from a flag. With declarations and a family whose runtime renders them, it asks the runtime for a structured call instead of reading text, twice: with constrained decoding off — the runtime's default, and what the reference is compared with — and on, which is what an application that enables it gets. How far the two differ, and which way, is reported. A prompt the runtime gives no reply to is scored as a wrong answer and counted apart by reason: its call parser rejecting what the model wrote, or a prompt reaching the bundle's token limit. Any other reason, on any prompt, leaves that mode unmeasured — with the grammar off the run ends as a harness failure, with it on the mode is reported as not measured and the grammar-off number stands — and anything else going wrong ends the run. The reference's text is read the way the runtime reads a reply, only between the call markers, and a reply with more than one call is a wrong answer on both sides.

What it refuses. Structured targets in runtime_rendered without --declarations: their descriptions and types are your application's contract and cannot be read off the targets. --declarations for a runtime_rendered split of a family whose runtime litetune does not record as rendering them. A structured target for a family whose call format litetune has not measured: supply each row's completion instead. In tune, a row with a target and no completion — run prepare, which writes it — and a call row whose completion the runtime would not read as exactly its target's call, with the target's types, and nothing after it: no call markers, as in a split prepared by 0.1.6 or earlier, a number written as a string, text after the call, or a stop token inside a string. Drop such a completion so prepare renders it. A call to a tool the declarations do not offer, and a call in a row whose target is text. Marked calls in runtime_rendered for a local checkpoint whose config.json says gemma3_text, which is FunctionGemma and Gemma 3 alike: say which it is in its litetune.json. verify without --declarations for a runtime_rendered checkpoint that recorded some, and --scorer exact-text on the tool path. In prerendered the declarations file's bytes are the record, so the file tune read has to reach verify and bundle unchanged. prepare without --base-model still renders FunctionGemma's format, and the report says that it assumed it.

Gemma 4 towers: drop them, or take them from another bundle

Google's Gemma 4 E2B CPU/GPU bundle carries its vision and audio towers as six sections of their own. An application that only sends text can ship without them:

litetune towers --model gemma-4-E2B-it.litertlm --drop vision --drop audio \
                --output gemma-4-E2B-it-text.litertlm

A section belongs to a tower by its section_type (a TFLite graph or its weights) together with a model_type from litert-lm-builder 0.18.0's own list for that tower; every other section is kept. The new file is read back before it is written to --output: every section, the metadata included, byte for byte against the section it came from. A file its own header does not fit -- cut short, or naming sections outside it or overlapping -- what the builder cannot write back (a section unpack leaves out, such as TTS or ASR metadata, or a model type it does not know), a tower the bundle does not carry and an existing output are refused. The work directory beside --output needs about the bundle's size plus the result's.

On Google's bundle the six tower sections are 332,387,932 of 2,588,147,712 bytes. The bundle litetune towers --drop vision --drop audio wrote from it answered eight greedy prompts on litert-lm 0.18.0's CPU backend exactly as the full bundle did, at a peak RSS within 1 MB of it, and every image or audio input to it through the Python API raised an error rather than producing text; opened with a vision backend, its engine is not created at all (MEASUREMENTS.md). What your model answers is verify's to say.

The other way round, towers from one bundle into another — Google's E2B towers into a text-only build compiled for a Qualcomm NPU:

litetune towers --model gemma-4-E2B-it-npu.litertlm --add vision --add audio \
                --from gemma-4-E2B-it.litertlm --metadata-from-donor \
                --output gemma-4-E2B-it-npu-towers.litertlm

The bundle's own sections stay byte for byte. The donor must carry each whole tower, and each adapter it gives must write as many values per token as the bundle's embedder, as the graphs' signatures say: LiteRT-LM copies the one into the other without comparing them. The metadata kept must be Gemma 4's and name, as a string, the token each added tower's input starts with, and every token_str and token id it names must be the same piece, of the same type, at the same id in both tokenizers. --metadata-from-donor replaces the bundle's whole LlmMetadata with the donor's -- its prompt template, its stop tokens, its model type, which selects the runtime's model-specific code paths, and every other field the two differ in, each the builder's text form shows listed in the report as old → new. It is refused when the two differ in max_num_tokens or kv_cache_init_value, which LiteRT-LM's NPU executor reads, or in the settings of a tower the bundle keeps, and required when the bundle's own metadata is not Gemma 4's or names no such token, as the text-only build's did not. Without it, the bundle's settings for the added towers must equal the donor's. With --add the work directory needs the donor's size on top. This is not an operation LiteRT-LM offers; it was measured on one SM8850 phone, through flutter_gemma's LiteRT-LM 0.18.0 C API and Qualcomm dispatch with QNN 2.51.0 (not the litertlm-android AAR's Backend.NPU, which was not tried), where what this command wrote answered an image and an audio turn and its text answers did not change; a turn carrying both was answered for the image alone, as Google's own bundles answer it through the same runtime (MEASUREMENTS.md).

What it knows that a shell script does not

Each of these was paid for once, by an artifact that looked fine and was not.

Export flags keyed on model identity. They are in no documentation, and config.json does not contain enough to derive them:

A LoRA scope keyed on model identity. On a multimodal checkpoint the vision and audio towers use the same projection names as the text layers, so --method lora scoped by name alone reaches all three, and peft refuses the tower ones rather than adapting them -- transformers wraps those in Gemma4ClippableLinear and peft dispatches on a bare nn.Linear -- so such a run stops before its first step. For the Gemma 4 families litetune restricts the run to modules under language_model and hands peft a regex rather than a name list, because peft matches a plain target_modules list by name suffix and a suffix cannot say "only under this container". tune reports it on the run, in plain output as well as --json, and convert --json and every verify manifest carry it under model_rules.rules.lora_container in convert's output and model_rules.lora_container in a verify manifest, which nest it differently. What the projection set is worth was measured once: the seven names litetune uses against the two peft's own Gemma 4 default would have chosen, 24 points apart on banking77, in MEASUREMENTS.md.

Without the first, FunctionGemma exports as a generic model — its config.json says gemma3_text, which the exporter does not recognise, so it falls through a silent catch-all. The runtime then builds no tool-call channel at all. An app that parses the response text sees nothing wrong; an app that passes tools natively receives no calls. The export succeeds, the file is the right size, every liveness check is green.

A trained checkpoint is told what it came from. tune writes litetune.json beside the model, and convert reads it. This is not bookkeeping: the per-family export flags key on the model's name, a directory has none, and transformers 5.x deletes _name_or_path from config.json on save — so without it a checkpoint this tool produced would export with none of the flags its family requires, successfully and silently. config.json cannot stand in: FunctionGemma and Gemma 3 270M/1B all declare model_type: gemma3_text and need different values. A checkpoint from elsewhere says so with --base-model or --train-metrics; one that says nothing is refused rather than guessed at, because guessing wrong ships a bundle with no tool-call channel that passes every check.

The prompt template has to be one the device can execute. FunctionGemma's own template uses macro and dictsort; LiteRT-LM renders with MiniJinja, which supports neither. A bundle carrying it exports cleanly, is the right size, passes every liveness check, and still answers a plain text prompt — then fails the native tool-call path, where LiteRT-LM routes the call through the chat template, with litert_lm_conversation_send_message_stream failed, which is the whole error the Python binding raises. The split is in the runtime, so every consumer that hands it tools sees it, whatever it is written in. litetune ships a template the runtime can run and passes it on export. Measured on the same checkpoint: with the override the runtime answers [tool_call] set_alarm{hour:7}; without it, INTERNAL: Failed to apply template.

From LiteRT-LM 0.18.0 a template is handed a list of parts. The runtime gives the bundle's template each message's content as [{"type": "text", "text": ...}] where 0.17.1 gave it the string, and a template written for a string may print the list, drop the user's text or fail: measured on 2026-10-08, a Qwen3 0.6B bundle rendered <|im_start|>user\n<|im_end|> with no error and every send to a Qwen2.5 0.5B Instruct bundle failed. For a checkpoint either rule claims -- a name that contains Qwen3-0.6B, Base included, or ends in Qwen2.5-0.5B-Instruct, under any organisation, a local snapshot of either and what tune made from either, but not a Hub repack named for its format, such as -FP8, -GGUF or -bnb-4bit, nor a local checkpoint whose config.json has a quantization_config -- convert reads the checkpoint's own chat template the way the exporter reads it, through transformers.AutoTokenizer in the export environment and, when the tokenizer has none, the checkpoint's chat_template.json. It writes that template unchanged behind a block that turns a list of text parts back into the string, to _litetune-chat-template-<digest>.jinja in the output directory, hands that file to every recipe, and reads each bundle's template back: a bundle that does not carry exactly that text is a failed recipe. convert then tries to rename it to model.litertlm.refused, so that nothing takes it for one that passed, and the result names the path the bundle ends up at, which is the original one if the rename fails. A bundle whose header cannot be read, or whose header or sections run past the end of the file, fails the same way for every family. A tokenizer with several named templates gets its default composed; the exporter left alone never selects default: it unpacks the dict -- that is, its keys -- into three values (litert_lm_builder.py:320), which raises unless there are exactly three templates and otherwise takes their names for prompt parts. convert refuses, and exports nothing, for a checkpoint with no template, one that is only whitespace, one with a carriage return (the exporter reads the file in text mode, which would turn it into a newline), and one the block is not for: before any export it renders one user message through transformers' Jinja2, as a string through the template alone and as parts through the block, and the two must print the same. That is transformers' renderer, not LiteRT-LM's MiniJinja; verify's rendering check is what checks the runtime. A template you pass yourself with --flag=--jinja_chat_template_override=<file> is used as given, with no block, so on LiteRT-LM 0.18.0 it has to read a list of parts itself, and convert says so in a note. A metadata override (--litert_lm_llm_metadata_override) is refused for these checkpoints, with your template or litetune's: the exporter puts it in place of the metadata it builds, which is the only way a template reaches the bundle. A tool response LiteRT-LM sends as a tool_response part stays a list, as on 0.17.1, and neither Qwen3-0.6B's nor Qwen2.5-0.5B-Instruct's template reads it as text, so tool use with these bundles is not covered.

Exporter flags are read once, as the exporter reads them. litert-torch export_hf parses its command line with Fire, and convert parses every --flag= the same way before deciding anything: - and _ in a name are one, --name alone is True and --no<name> alone is <name>=False, a value is the Python literal Fire makes of it (None, 0, '', a quoted string), and of a repeated name the last wins. The exporter is then handed one argument per name, the one in effect, so what convert reports is what the exporter acts on. Refused, each with what to pass instead: an argument Fire would take as positional (no leading hyphen and letter), a bare --, a value in the next argument (give it in the same one, --flag=--name=value), and --model, --output_dir or --quantization_recipe, which convert passes from its own --model, --output-dir and --recipe.

A bundle of a checkpoint these two rules claim needs converting again, unless it was converted with a template of your own, when:

• litetune 0.3.0 or earlier converted it: it carries the checkpoint's template alone;
• litetune 0.4.0 converted it under a name 0.4.0 did not claim: 0.4.0 claimed only Qwen/Qwen3-0.6B, Qwen/Qwen3-0.6B-Base and Qwen/Qwen2.5-0.5B-Instruct -- as Hub ids, local snapshots, paths ending in one of those names and what tune made from them -- so another organisation's copy, or a fine-tune under its own name, carries its template alone too.

verify refuses a bundle of either kind at its rendering check and names this as a possible cause.

A bundle litetune 0.4.0 converted from a checkpoint it claimed carries the template of Qwen/Qwen3-0.6B at c1899de or Qwen/Qwen2.5-0.5B-Instruct at 7ae5576 behind the block, whatever the checkpoint's own was. For those two checkpoints and what tune made from them that is their own template, and converting again gives the same bundle but for Jinja comments (MEASUREMENTS.md, the 0.18.0 section). Qwen3-0.6B-Base and its fine-tunes got Qwen3-0.6B's: rendered by transformers' Jinja2 on 2026-10-12, Qwen3-0.6B-Base's template at da87bfb and Qwen3-0.6B's at c1899de print the same prompt for one user turn and for a two-turn text conversation, and differ only where a message carries a tool response, reasoning content or content that is not a string. Other revisions of either repository were not compared.

Any other family's template is not rewritten: convert says so in its note on a checkpoint it has no rules for, and verify's rendering check is what shows whether it reads a list.

The terminator comes from the chat template, not from eos_token_id. They are not always the same token, and a model trained to emit the wrong one never closes its turn — on a device it emits call after call, and a consumer that delimits the reply by the turn marker cannot find the end of one.

The adapter is saved before the merge. A merged checkpoint cannot be un-merged, so the rank-16 delta is the only form you can re-apply to a different base, inspect, or ship on its own.

tokenizer.model is carried back. transformers 5.x stopped writing it and the tokenizer classes stopped exposing vocab_file, so the exporter's SentencePiece branch never fires and the bundle silently gets an HF tokenizer section — losing FST-constrained decoding, which is SentencePiece-only. tune copies the file back and records in metrics.json whether it managed to.

metrics.json records which device trained the checkpoint. tune resolves the device once, before training starts, and writes it to device as "cuda", "mps" or "cpu" — the field is absent only in a metrics.json written by a version of litetune that predates it, never a guessed value. It is the durable answer to where a given checkpoint was trained. Beside it, device_probe says who chose the device (the probe, or LITETUNE_DEVICE) and what torch reported, host names the operating system, its version, the machine and, on macOS, the chip, dtype_source says whether the dtype was declared or the default for that device. batch_size, micro_batch_size, micro_batch_size_source and gradient_accumulation_steps record the effective optimiser batch and how its forwards were split. mps_environment holds the three MPS variables as the training script read them. mps_memory, pad_to_multiple_of and mps_memory_sampled_max are null off mps. On mps, mps_memory records who set the limit, litetune's computed budget beside the limits in force, the recommended working set, both watermark ratios and who set them, the memory reading and swap use at the start; mps_memory_sampled_max is the largest of torch's current_allocated_memory() and driver_allocated_memory() over samples taken after each forward pass, backward pass and optimiser step — the largest value at those points, not a peak — and a script that did not see the variables litetune sent is a limitation. The reference side records the same probe, host and memory policy in the verify manifest's reference engine, with the counters sampled after each generate call.

Minimum transformers per family. Gemma 4 and Qwen3.5 fail at tokenizer load on every 4.x release, and Gemma 4 needs 5.5.0 for AutoConfig to recognise the architecture. litetune refuses with the version rather than letting you find out from an AttributeError.

One environment per stage. The training stack and the export toolchain pin incompatible dependencies and cannot share an interpreter.

Results

Every converted-model figure in these tables was taken on litert-lm 0.16.1, the runtime litetune 0.1.x pins; 0.2.0 – 0.3.0 pin 0.17.1, and 0.4.0 – 0.4.1 pin 0.18.0. MEASUREMENTS.md has the 0.17.1 re-measurements beside the originals where there are any, and the same again on 0.18.0.

functiongemma-270m-it, LoRA on google/mobile-actions, scored on 640 examples the model never trained on:

Your gain from fine-tuning depends on your data. What this table is here to show is the last row: conversion cost something, it was small, and one of the two figures is not distinguishable from zero at this sample size.

The two artifacts are 0.04% apart in bytes. Nothing in file size, exit code or logs separates them — running both against held-out data is the only thing that does.

gemma-3-270m-it, LoRA on mteb/banking77 (77-way intent, --scorer exact-text), scored on 600 examples the model never trained on:

A different family, a different scorer, and this time neither conversion figure clears its interval — the dynamic recipe lands on the same score as its float twin while still disagreeing with it on 34 of 600 prompts. The base row says refused because it was: asked for one intent label, the untuned model returned nothing at all on 53 of 600 prompts, and verify stops at the liveness tier rather than scoring a model that never answered. An earlier run of this same pair, measured in the other prompt mode, is what found the exact-text terminator bug fixed in 0.1.5 — see MEASUREMENTS.md.

Qwen3-0.6B, LoRA on the same 2,400 rows, scored on the same 600:

The first family measured here that litetune had no rule for. It exported with no flag from litetune; from LiteRT-LM 0.18.0 its rule puts a block in front of the checkpoint's own template, for the runtime rather than the exporter. The weight-only figure clears its interval here where the dynamic one does not, and where neither of gemma-3-270m's did — on 12 disagreements out of 600. Training and the float reference ran on a GPU and the converted models on a CPU, so this cost carries a hardware difference as well as a conversion one — as the Gemma 3 one does too; every manifest in both runs records it — see MEASUREMENTS.md. On litert-lm 0.17.1 the same bundle's weight-only cost is +0.0050 on 5 disagreements and does not resolve; which verdict holds is not established.

MEASUREMENTS.md has the intervals, three runs of the same configuration and what they disagree about, and which published claims were withdrawn after re-measurement.

Limitations

Known to be broken

• The model does not always stop on the device. 8 of 640 responses ran to the token limit — two carried 350 and 351 identical calls — and 13.4% carried more than one call, against 5% on the cloud CPU. The bundle names <end_of_turn> and <start_function_response> as stop tokens, so whatever stops the float path is not stopping this one. Not diagnosed.
• Peak memory is not bounded. Training this model on 8,693 examples was OOM-killed at 32 GiB more than once. There is no preflight check; a death with no Python traceback is probably this.
• A Gemma 3 270M NPU bundle needs cache_length ≤ 896 at prefill_128 on SM8750 and SM8850; above that it keeps only the first prefill chunk. Google's published 270M and 1B sm8750 bundles on a Galaxy S25, its 270M sm8850 bundle on a Galaxy S26 and every litetune build at cache_length 1024 lose everything after the first 128-token chunk: a 245-token prompt with the question at the end is answered as if the question were not there, and a tool prompt is garbage — reported and bisected from this project as LiteRT-LM#3508. On SM8750 the loss tracks the compiled prefill graph's attention mask crossing about 1.0 MiB — 128 × (cache_length + 128) × 4 B × 2 for a prefill_128 export, the size defect in litert-torch#1184. Every bundle that failed in the issue is above that line; the one published bundle that works on SM8850 (1B, 1.375 MiB) is above it too, so the rule is a measured boundary for the 270M graphs on both SoCs, not yet a traced cause. The boundary is per model, not a number to carry across. Reading the SRQ graphs, the operand of the failing add is the int16 mask tiled over the query heads inside the compiled prefill graph — 2 B × num_attention_heads × prefill × (cache_length + prefill) — which for Gemma 3's four heads is the same number as the count above over the bundle's two fp32 mask inputs, and for a sixteen-head model is not: Qwen3-0.6B is 4.0 MiB at cache_length 896, still above the line (LiteRT-LM#3508, 2026-09-11). At 896 the mask is exactly 1.0 MiB and all five chunks of a tool prompt survive; 768, further below the line, keeps the context too (checked on two-chunk prompts). So the FunctionGemma NPU bundle is prefill_lengths = [128], cache_length 896: about 640 tokens of declarations plus request, and about 250 for the reply.
• A Qualcomm NPU bundle loads only in an app whose QAIRT is at least as new as the compiler's. The public ai-edge-litert-sdk-qualcomm 2.2.0 writes QAIRT 2.47 context binaries. An app whose QNN libraries are QAIRT 2.44 (what the flutter_gemma example app loaded on 2026-09-07) fails with Failed to create engine; the logcat line is Context binary (2.47.0) is newer than the current SDK (2.44.0). The native-v0.16.0 runtime tarball from the same project carries 2.47 and loads Google's 2.44-built bundles as well as ours; it is the runtime behind the S25 numbers in this file. On LiteRT-LM 0.18.0 through native-v0.18.0-a's dispatch, a 2.47 context binary was not enough: QAIRT 2.47 libraries created no engine on an SM8850 phone, and QNN 2.51.0's did (MEASUREMENTS.md, Towers grafted into an SM8850 NPU bundle). The Maven litertlm-android 0.16.1 AAR cannot reach the Qualcomm NPU with any public dispatch library (LiteRT#6889).
• An Intel NPU keeps only the first prefill chunk, and not for the Qualcomm reason. The dynamic_wi8_afp32 graph of the same export (the OpenVINO compiler rejects the static-int8 stage that follows it), compiled for Lunar Lake with ai-edge-litert-sdk-intel 2.2.0 and run on a Core Ultra 7 258V through the LiteRT-LM C API, answers single-chunk prompts 20/20 and loses every multi-chunk prompt regardless of cache_length — byte-identical output at 896 and 1024. So the mask-size rule is Qualcomm's alone; Intel's cause is not found. As of 2026-09-08 there was no Google Gemma 3 Intel bundle to compare against.
• Gemma 4 cannot be built for an NPU with the public exporter. On the transformers this tool pins (5.16.1; ≥ 5.14 removes global_head_dim) the split-cache NPU export in litert-torch-nightly 0.10.0.dev20260826 fails before the graph, and a bundle built on transformers 5.13.1 loads on the Snapdragon 8 Elite and returns garbage: Google's published Qualcomm Gemma 4 bundle carries a per-layer-embedder layout the public exporter does not produce, and the runtime's NPU path expects that layout. The regular CPU/GPU export of Gemma 4 completes on the same pin, and a CPU one now has a conversion-cost figure — see MEASUREMENTS.md.

Limits on the numbers

• Some of them are litert-lm 0.16.1's. The four banking77 eight-bit pairs, the four-bit recipes on the same four checkpoints, the tuned Gemma 4 E2B's eight-bit pair and FunctionGemma's tool path were re-measured on 0.17.1, which 0.2.0 – 0.3.0 pin, and again on 0.18.0; among what was not are the headline FunctionGemma runs, the conversions of Gemma 4's base weights, the runs on untuned bases and the phone runs. MEASUREMENTS.md says which is which.

• Measured on six models, all six fine-tuned here. functiongemma-270m-it with the tool-call scorer, and gemma-3-270m-it, gemma-3-1b-it, Qwen3-0.6B, Qwen2.5-0.5B-Instruct and gemma-4-E2B-it with exact-text, each with a conversion cost against its own float twin. Only FunctionGemma also has a training gain: no banking77 run has an untuned base figure to subtract — Gemma 3 270M's base was run and refused to score, the 1B's and Qwen2.5's scored 0.0000 on both sides, Qwen3's never reached the base step at all, and Gemma 4's tuned run records no base figure either — and every one of those manifests records the gain as unavailable. gemma-4-E2B-it carries a second, earlier run as well: two conversions of its untouched base weights against the float reference, on prompts that list all 77 labels rather than the prompts the tuned runs use, so that figure is not comparable with the five above it. Qwen3.5 exports but has no quality figure, and a Gemma 4 without its variant is still refused unless you supply the template override yourself.

• The turn-terminator vocabulary is a static list. exact-text scoring and the liveness checks both trim against a fixed set of strings, recorded verbatim at harness.terminators in every verify manifest. A family whose chat template closes a turn with a marker outside that list would otherwise have its reference score at zero, which is a fact about the vocabulary rather than the model — so verify stops before scoring either side and reports a harness failure instead of a conversion cost. That is the safe direction, not a fix: the run still cannot be measured until the vocabulary knows the marker. Gemma 4 is the case that proved it — it closes a turn with <turn|>, all 600 reference generations ended there, and the comparison was refused until the vocabulary learned the marker. Resolving it from the bundle contract instead of hardcoding is a follow-up. To find your own model's marker before then, tune records it at turn_terminator.text in metrics.json and bundle carries it into contract.json's stop_tokens.

• In runtime_rendered, verify refuses to compare two sides that were shown different prompts. Before generating anything it renders every held-out prompt through the runtime's own conversation path and through the reference's chat template, and compares the token ids; on the first 8, or on all of them if the split is shorter, it also compares the prefill count the runtime reports when the prompt is actually sent. Any difference is a harness failure (exit 4) with the prompt, both counts and the first differing position at harness.rendering_check, not a conversion cost. On every tuned export of both families all 600 banking77 prompts matched, and on the gemma-3-270m-it base export too; a rendering that adds an empty <think></think> is refused on every prompt, which is constructed in the tests rather than seen in a run. Reasoning is removed from both sides before scoring, through the last [/thought] or </think>, and counted per side at measurements.<side>.reasoning_removed, including generations that never closed it.

• The candidate runs on CPU unless you pass --backend gpu; the reference resolves its own device, and your users run on a phone. On one Snapdragon Galaxy S24 (SC-51E), the dynamic_wi8_afp32 bundle on the device's CPU scored 0.8703 ±0.026 on the 640 held-out rows against 0.8906 for the cloud CPU run that produced it (run A in MEASUREMENTS.md; runs B and C scored 0.9016 and 0.8969, both just outside that interval). So the reference number predicted the phone to within about 0.03. One device, one recipe, and not re-measured since the runtime moved to 0.17.1.

• On a GPU box or a Mac on mps, the reference and the candidate can run on different hardware, and it is recorded rather than refused. The candidate runs on the litert-lm backend --backend names, cpu by default, and the reference resolves its own device; where the reference lands on cuda or mps, the two sides differ in hardware as well as in conversion. harness.device_mismatch in the manifest names both devices and both engines, and the same text is carried into the run's limitations, so the conversion-cost number does not silently carry a hardware difference too. To put the reference on the CPU beside a CPU candidate, set LITETUNE_DEVICE=cpu.

• The GPU number is 20 rows. Same device, same bundle, GPU backend: 20/20 tool names and 15/20 exact (CPU: 20/20, 14/20) at 1.8× the CPU speed — with prefer_activation_type = fp32 in the bundle. Without it the GPU text executor computes in F16 and returns <pad> floods and invented tool names (3/20), while the engine reports success. convert writes that key into every bundle it produces that does not already declare one; --json records what each carries as exports[].gpu_activation, and a bundle that could not be repacked is named in the limitations: an app that loads it without an activation override gets the F16 default. litetune cannot drive a phone GPU from a laptop, so a device run is a separate job.

• verify --backend gpu measures on this machine's GPU, with fp32 passed by litetune. The activation type is passed on every GPU run, so a bundle without the key is measured rather than flooding — upstream calls the option experimental. On an M4 Pro's Metal with litert-lm 0.16.1, the FunctionGemma bundles above, 40 held-out rows through the tool path with grammar off: a bundle without the key answered identically to one with it on all 40 rows, and the same bundle with the override withheld flooded <pad> on all 40. Because that number is for the bundle plus the override, verify reads the bundle's own key and says so in the manifest when it is not fp32. GPU and CPU still do not agree row for row: the function name matched on all 40, the whole first call on 29, and every difference was a free-form date. litert-lm reports no device back, so litetune asks the kernel: on macOS a GPU engine's process owns a GPU user client, and the run is recorded as measured on the GPU only when that client's GPU time grew while it generated — measured on the same M4 Pro, where a CPU engine owned none. A run the kernel shows doing no GPU work is said, in the limitations, not to have used the GPU. Off macOS nothing is looked at yet, and the manifest records the GPU as asked for. On a host without a usable GPU, a GPU engine has been measured to produce nothing, so a GPU split on the text path first runs one prompt within one prompt's budget and stops there if it gets no answer. The tool path has no such check yet: there each decoding mode waits out its own budget, an hour.

• The NPU number is 20 rows on one SoC. The same functiongemma-270m-it (base weights, not the fine-tune) through litert-torch's npu_export stages for a Snapdragon 8 Elite (SM8750, Galaxy S25), on the phone's NPU through the LiteRT-LM C API, greedy, cache_length 896. Five-chunk tool prompts of 579–635 tokens produced a parseable call on 13/20 rows, the right tool name on 12/20 and an exact match on 3/20, at a median 441 ms per prompt including that prefill. The same static-int8 graph on the CPU interpreter scored 20/20, 18/20 and 12/20, so the nine exact rows the NPU loses are its own decode: one to syntax the runtime's parser rejects (a stray }), one to a prose reply instead of a call, one to the wrong tool named first of two calls, six to a wrong or missing argument in an otherwise well-formed call. Not diagnosed beyond that; not yet measured on the fine-tuned weights. The full table across prompt sets is in MEASUREMENTS.md.

• Two prompt renderings are in the field for the same model, and they disagree for every declaration with more than one property. Costly on a base checkpoint, near-free after fine-tuning; contract.json records which you used. See MEASUREMENTS.md.

Not built yet

• Nothing here configures the app that loads the bundle. An app targeting API 31+ must declare <uses-native-library android:name="libOpenCL.so" android:required="false"/> (plus the -pixel/-car names the loader also tries) or the runtime reports "Can not find OpenCL library on this device": in that case it is the missing declaration, not the device — though the same string covers devices with no public OpenCL at all. Set an output cap and a maxNumTokens: the bundle's KV cache is 4096 and a run that does not stop fills it (13 of 20 GPU rows ran to 3,500 <pad> tokens and 53 s each before the fp32 fix; capping at 1024 cut that to 6 s by cutting the garbage, not by fixing it). And give EngineConfig.cacheDir a writable directory.
• Decoding parameters reach only one side. litetune passes none to the device, so the reference is held to an explicit token limit while the device runs to the runtime's own. The manifest says so and counts unterminated generations.
• convert does not compile for an NPU. The NPU bundle described under Known to be broken was built by hand: the four functions in litert-torch's generative/export_hf/experimental/npu_export/stages.py — npu_export, npu_calibrate on 64 tool prompts disjoint from the scored set, npu_quantize, npu_compile — with the compile on Linux x86_64 through ai-edge-litert-sdk-qualcomm 2.2.0, one compile per SoC, cache_length 896. The device outputs came from a C-API harness that is not in this repository and were scored outside verify (litetune's call parser plus a short comparison script); verify has no device mode. litert-torch's documented one-step export-hf --aot_backend=qualcomm is not this path: its float bundle of gemma-3-270m-it loads on the HTP and is garbage on every row, single-chunk included. Until the stages are in convert and the fine-tuned figure exists, "NPU" here means the four NPU bullets under Known to be broken, not a flag.
• .litertlm only, no library API, and no single run command.

Exit codes

The verdict is the exit code; the printed summary renders it. A run that completed and a run that could not be judged must never be confused, which is why there are five and not two.

4 is not a failure of the model. It means no answer could be obtained: a missing shared library, a malformed input, a killed process — or a command line litetune would not run. That last one is why a usage error exits 4 and not argparse's usual 2: here 2 means inconclusive, which is a claim about a measurement, and a typo should not produce one.

bundle carries a verdict rather than producing one, so it returns whatever --status or --verify-manifest gave it. With neither it returns 2: bundling re-measures nothing.

towers measures nothing either: it exits 0 when it wrote the new bundle and 4 when it refused or could not, or when it wrote the bundle but could not deliver the --json report.

Wiring || exit 1 on anything non-zero throws all of this away.

Contributing

Issues and pull requests welcome, particularly measurements on models other than the six above — that is the gap this alpha most needs closed.

Run the checks with pytest, ruff check, ruff format --check and mypy src.

scripts/ci-local.sh runs what CI runs on Linux, on the platform CI runs it on: the three interpreters in Linux containers, then the wheel job. CI's macOS job needs a Mac with uv 0.10.6; the script's header gives its commands. It needs podman or ENGINE=docker, and writes nothing to your working tree. Worth the minute before sending anything that touches processes, signals or paths — a green suite on macOS and a red CI run happened on the same commit twice, because os.waitid is absent from CPython there before 3.13 and the same test took a different branch on each.

License

Apache 2.0.