loom-py

import loom

model = loom.Model.from_pretrained("loom-ai-org/gemma-3-270m-it-loom")
print(model.text2text.chat("Who discovered Brazil?"))
# 'The discovery of Brazil was made by **Hernán Cortés**.'   (one sample: this
# checkpoint declares do_sample, so pass temperature=0.0 for the same answer twice)

Two APIs, and which one you want

The high-level API is one door per task, named for the modality pair it maps between. Every model carries all of them; the ones it does not answer to say so when called, naming what it actually is.

model.text2text.infer("The capital of France is")        # -> str, a raw completion
model.text2text.chat("Who discovered Brazil?")           # -> str, asked inside a turn
model.speech2text.infer(waveform, language="en")         # -> Transcription
model.text2speech.infer("hello world")                   # -> Audio
model.text2class.infer("Wolfgang lives in Berlin")       # -> Classification, a label per token
model.text2codes.infer("[S1] Hello world.")              # -> codec tokens, frame-major
model.codes2speech.infer(codes)                          # -> Audio, the other half of that pair
model.speech2class.infer(waveform)                       # -> AudioClasses, per clip or per frame
model.speech2embeddings.infer(waveform)                  # -> list[float], one vector per clip

Which door a model answers is read off the file, not guessed from its name:

model.task            # 'automatic-speech-recognition'
model.capabilities    # ('speech2text',)
model.text2speech.infer("hi")
# UnsupportedTask: this model is speech2text (automatic-speech-recognition), not text2speech.

Each door does the whole job, including the parts you cannot do from outside. speech2text windows audio for a model whose graph is built at one clip length, decodes with the early stop armed, splits the output into timestamped segments and seeks to where the model closed its last segment — so an utterance straddling a window edge is re-decoded whole rather than arriving as two fragments:

r = model.speech2text.infer(audio, language="en", timestamps=True)
r.text                        # the joined transcript
r.segments[0].start, .end     # seconds, whole-file
r.timestamped                 # whether those are boundaries the model chose

text2speech returns audio with its rate attached, because a bare list of floats played at the wrong rate does not fail — it plays at the wrong speed:

audio = model.text2speech.infer("hello world", steps=8, seed=1)
audio.sample_rate             # 22050
audio.save("out.wav")

text2class returns the labelled pieces beside the label set they came from, because neither is recoverable from the other — a WordPiece encode splits words, so joining the pieces back up is a rule only you can make:

r = model.text2class.infer("Wolfgang lives in Berlin")
r.labels                      # ['O', 'B-MISC', 'I-MISC', 'B-PER', ...] — every class the model has
[(t.piece, t.label) for t in r]
# [('Wolfgang', 'B-PER'), ('lives', 'O'), ('in', 'O'), ('Berlin', 'B-LOC')]

The framing tokens the encode added ([CLS]/[SEP]) are dropped for you, on the ids the file declares rather than on their spelling; strip_special=False keeps them if you want the raw alignment.

speech2class returns the model's own distribution, not a decision — a VAD is thresholded and smoothed differently by every consumer, and a language id is read top-k — with the time each frame row starts at, read off the file:

r = vad.speech2class.infer(audio)            # one row per 20 ms frame
r.granularity, r.labels                      # 'frame', ['non_speech', 'speech']
speech = r.probability("speech")             # the speech curve
r.times[:3]                                  # [0.0, 0.02, 0.04] — seconds, frame starts
lid.speech2class.infer(audio).top(3)         # [('th: Thai', 0.988), ('lo: Lao', 0.012), ...]

speech2embeddings returns the model's vector unnormalised; comparing two (cosine, for a speaker embedding) and choosing a threshold are yours.

A model that speaks through a codec is two files, and the codes are what joins them. An autoregressive codec LM emits discrete tokens, not audio; a neural codec turns those tokens into a waveform. They stay separate because one codec serves many LMs, and because the codes are worth having on their own — cache them, stream them, decode them at a different rate:

codes = dia.text2codes.infer("[S1] Hello world.")   # -> [[568, 778, ...], ...] frame-major rows
audio = dac.codes2speech.infer(codes)              # -> Audio

Nothing goes between the two calls. Both files declare loom.codec.n_codebooks, so a pair that does not fit says so instead of producing audio of the wrong duration:

dia.hparam("codec.n_codebooks", "u32") == dac.hparam("codec.n_codebooks", "u32")

text2codes counts max_new_tokens in audio frames, not decoder rows — the two differ by the model's delay pattern, which is an artefact of how codebooks are written rather than anything you asked for.

The low-level API is the driver's own entry point, and stays raw. infer passes your arguments straight through, so which ones a model takes is a property of the model rather than of this package:

model.tokenize("The capital of France is")   # [1, 1098, 5706, 803, 4481, 856]
model.detokenize([1, 1098, 5706])            # '<|startoftext|>The capital'
model.infer(tokens=[16, 40, 22, 30], n_steps=4, seed=1234)   # the driver's own inputs
print(model.driver_source)                   # the Lua that will run, and what it accepts

Use it when you want a knob the high-level door does not name — VITS's noise_scale_w, a specific voice vector, a model's own second entry point. That is the boundary between the two: a knob with no canonical role is reachable through infer and nowhere else.

Text to speech, and the one step that is not in the file

Every TTS model here takes text now, but two of them get there differently. Supertonic encodes graphemes itself. The other four consume phoneme ids — and the symbol table that turns phonemes into their ids ships in the GGUF, so model.tokenizer is a real vocabulary for all of them:

model.tokenizer                      # <loom.Tokenizer 'phonemes' size=159>
model.tokenize("h\u0259\u02c8lo\u028a")             # -> ids, with the model's own BOS/blank/EOS assembly

What is not in the file is grapheme-to-phoneme, because that is a property of the language rather than of any checkpoint. It is an optional extra:

pip install "loom-py-rt[phonemes]"

With it, text2speech.infer("hello world") works on all five. Without it, passing phonemes= or tokens= works exactly as before and only the text door is absent, with an error naming the install. loom.phonemizers.register("ipa", fn) substitutes your own.

Choosing a device

A wheel built with a GPU backend uses it by default; one built without has only a CPU to find, so nothing changes.

model = loom.Model.from_file("qwen3.gguf")                  # decide for me (or $LOOM_DEVICE)
model = loom.Model.from_file("qwen3.gguf", device="cpu")    # pin it
model = loom.Model.from_file("qwen3.gguf", device="gpu")    # demand one; raises if there is none
model.device, model.device_description   # ('Vulkan0', 'AMD Radeon Vega 3 Graphics (RADV RAVEN2)')

"gpu" raises rather than falling back, because a caller who spelled it out is asking a question about the machine and a silent CPU run is how a large slowdown goes unnoticed. "auto" — the default — is the one that falls back.

The base wheel is CPU-only, and an accelerator is a separate install rather than a different wheel:

pip install "loom-py-rt[vulkan]"

That adds one small package holding one libggml-vulkan.so, which this package finds at import; device="auto" then uses it and nothing about the base wheel changes. The reason it works this way — rather than a full wheel per accelerator, which is the more familiar shape — is that a Vulkan backend is 46.5 MB and CUDA is larger, so the per-accelerator matrix does not fit PyPI's 100 MB per-file ceiling. See packaging/README.md.

loom.devices()   # [{'name': 'Vulkan0', 'description': 'AMD Radeon Vega 3 Graphics (RADV RAVEN2)', ...}]

Worth calling after installing one, because a backend whose driver is too old — or which finds no supported device — loads without error and registers nothing, and the only other symptom is a model running at CPU speed. Note that with this build every backend is loaded at run time, the CPU included, so an empty device list means no backend library was found at all rather than no accelerator.

Which ops fall back to the CPU, and why some always will, is documented in loom.cpp's own build notes.

Why there is so little API

A loom GGUF carries its own graph topologies and its own driver script alongside its weights, so this package contains no per-architecture code at all. Loading a model registers whatever topologies the file declares and attaches a KV cache to the ones that say they need it; running one calls the driver the file shipped with. A model this library has never heard of works the day loom-exporter can produce it.

That is also why infer takes **kwargs: its arguments are the driver's arguments, and which ones a model takes is a property of the model. model.driver_source prints the Lua that will run, whose header comment documents its inputs — that is the authority.

model = loom.Model.from_file("granite_speech_mil.gguf")
model.architecture          # 'granite-speech'
model.topologies            # ['encoder', 'embed', 'decoder', 'lm_head']
model.hparam("samples_per_chunk")   # 192000
print(model.driver_source)  # what infer() will run, and what it accepts

Supported models

Twenty-two, published at huggingface.co/loom-ai-org and loadable by id with from_pretrained (needs the [hub] extra). This package has no per-architecture code, so the list is a property of loom-exporter, not of anything here.

Language models

These are the models text2text answers to; model.generate(...) is the same call under the name it shipped with.

infer and chat are different questions. infer CONTINUES a prompt, which is what a base model does; chat puts the prompt inside a turn using the checkpoint's own chat template and asks for a reply. An instruction-tuned model handed a raw prompt behaves like a base model and continues in the prompt's own format, which reads exactly like it is repeating you back — so use chat for the instruction-tuned rows above, and infer for the base ones. model.chat_roles says which a file is: empty means it carries no template, and Gemma 3's is ['user', 'assistant'] because its template folds a system message into the first user turn rather than emitting a block for it.

Decoding follows the checkpoint's own generation_config.json — Gemma 3 ships top_k 64, top_p 0.95 and is sampled; everything else here is greedy. Name temperature, top_k, top_p or seed to override it.

Text to text, through an encoder-decoder

The same model.text2text.infer(...) door as the models above — a host asking for text and getting text back should not have to know whether one stack or two produced it — but the advice about chat does not apply. This one is instruction-tuned and carries no chat template: the instruction is part of the text, as in "translate English to German: ..." or "Answer the following question. ...". model.chat_roles is empty, which is the file saying so.

Speech recognition

model.speech2text.infer(waveform, language="en") — the mel frontend is inside the graph, so a raw waveform is the input, and long audio is windowed and seeked for you.

Speech synthesis

All five take text through model.text2speech.infer(...); the four phoneme-input ones need the [phonemes] extra for the G2P step (see above), and every one of them accepts phonemes= or tokens= without it. Kokoro and Supertonic ship a default voice, so a published file speaks on its own. model.driver_source is the authority on what each driver accepts.

Token classification

model.text2class.infer("My name is Wolfgang and I live in Berlin") — one label per token, and the labels come back beside the PIECES the tokenizer produced rather than beside your words, since a vocabulary splits some of them. result.labels is every class the checkpoint can choose between.

Both answer through that one door and what differs is entirely what a label MEANS. The NER row's classes are entity tags, so the labels are read as SPANS (B-PER I-PER is one person). The punctuation row's are the mark that follows the token across twelve languages, so they are read back into a SENTENCE — feed it text with the punctuation already removed, and take each word's mark from its LAST piece.

Text to codec tokens, and the codec that decodes them

These two compose, and that is the point of the pair: model.text2codes.infer(...) turns a sentence into codec tokens, and model.codes2speech.infer(...) on the codec file turns those into a waveform. Neither is a speech model on its own — the first emits integers and the second takes them — which is why they declare audio_codes rather than audio on the side where they meet.

The three repos

Installing

pip install loom-py-rt           # once published -- `loom-py` on PyPI clashes with `loompy`,
                                  # and `loom-engine` normalizes to the already-taken `loomengine`
pip install loom-py-rt[hub]      # + from_pretrained()

Supported platforms

CPython 3.10–3.13, on:

There is no per-CPU choice to make. The wheel ships every libggml-cpu-*.so variant ggml builds for the architecture and picks one at import by scoring each against the CPU's own feature flags, so the same file serves the oldest and newest machine on its architecture. loom.devices() is how you check it worked.

Why macOS 14 and not 11. It is not an arbitrary floor and it is not the newest thing that happened to build. ggml's BLAS backend is compiled against Accelerate's new BLAS interface, whose symbols arrive in macOS 13.3 — and that backend is worth 1.80x on whisper-small, so dropping it was the worse trade. An 11.0-tagged wheel would install on macOS 11 or 12 and then quietly have no BLAS, because a backend that fails to dlopen is skipped silently. The floor is the honest version of what the binaries already require — 14 rather than 13.3 only because a wheel tag carries no minor version above macOS 11, so 13.3 is not a thing a wheel can say.

Apple Silicon and Apple Intel are not equally supported, and the table means two different things by "covers". The arm64 wheel is built, installed and exercised on a real M1 Pro — the apple_m1 rung, which is the lowest of ggml's three Apple rungs and so the one every arm64 wheel has to serve. Apple Intel is CI-only: it is built and imported by a runner and nobody here owns the hardware, so treat it as best-effort rather than as parity.

Metal is a separate package, not part of the macOS wheel — pip install "loom-py-rt[metal]". It is only 0.87 MB, so that split is not about size: a GPU outranks the CPU when you let the library choose a device, and on unified memory that ranking is not reliably right. Measured on an M1 Pro, whisper-small is 1.76x faster on Metal and VITS is 1.79x slower — so which way it goes still depends on the model, and asking for the package is how you opt into that trade. loom.devices() shows what you got.

Raspberry Pi. Every model in the range is covered, by one of two different routes, and which one you are on is uname -m.

aarch64 — a Pi 3, 4, 5 or Zero 2 W on 64-bit Raspberry Pi OS — is a plain pip install loom-py-rt and needs nothing else. This is the route to prefer wherever the board can take it: a Zero 2 W is ARMv8 hardware, and running it on the 32-bit image gives up NEON and every __aarch64__ optimisation in the engine for nothing.

armv6l — a Pi Zero, Zero W or Pi 1 — has a wheel too, and it is not on PyPI:

pip install https://github.com/loom-ai-org/loom-py/releases/latest/download/loom_py_rt-1.0.0rc14-cp311-cp311-linux_armv6l.whl

PyPI accepts only manylinux* and musllinux* platform tags for Linux and the manylinux policy's floor is armv7, so linux_armv6l is refused at upload — there is nothing to fix and no index to wait for. The wheel is a GitHub release asset instead, built on every release in an emulated Raspbian (.github/docker/Dockerfile.armv6) and checked under QEMU_CPU=arm1176, which is the Zero's own core. Verified on a real Pi Zero W, not only under emulation.

It is slow, and the Pi 4 numbers on this page do not transfer. An ARM1176 is one ~1 GHz core with VFPv2 and no SIMD at all, so this rung runs the generic-C kernels: nothing in libggml-cpu.so here is vectorised, because there is nothing to vectorise with. The 512 MB of RAM is the harder limit — a small TTS model at Q4_0 is the shape that fits; the 0.6B ASR and LM models are not.

armv7l — a Pi 2/3/4 on 32-bit Raspberry Pi OS — builds by the same path but has no published wheel and no board here to verify one on, so pip finds nothing and falls back to an sdist build. Install the 64-bit OS instead; the hardware is ARMv8 either way.

Which variant a board loads, verified under QEMU on every release by raspberry-pi-check in .github/workflows/wheels.yml:

Nothing extra is installed on a Pi: the CPU is the backend. [vulkan] does resolve on aarch64 and a Pi 5's V3D exposes Vulkan through Mesa, but no measurement here says that is worth doing — the CPU path is the supported one.

Windows has no wheels, and a source install does not currently work there. macOS does: both Apple architectures build from a checkout as well as from a wheel — see Supported platforms above. (Epic-08)

From a checkout — note --recursive, since the engine is a submodule:

git clone --recursive https://github.com/loom-ai-org/loom-py
cd loom-py && pip install -e .

No runtime dependencies. Arrays cross the boundary as plain sequences of floats, so numpy is something you may use rather than something this package makes you install — list, array.array, numpy arrays and torch tensors all work.

Testing

pytest tests/ci      # the Python layer: coercion and error paths. No model. What CI runs.
pytest tests/gate    # a real exported GGUF, end to end.

export LOOM_TEST_MODEL=~/loom-fixtures/matcha_mil.gguf
export LOOM_TEST_MODEL_INPUTS='{"tokens":[16,40,22,30,12,3],"n_steps":4,"seed":1234}'
pytest tests/gate -q

The gate suite is written against no particular architecture on purpose: it asserts the shape of what a loom model is, which is the whole of what this package knows. A test that expected one model's inputs would be this package learning about a model, which is the thing the design exists to avoid.

Roadmap

Shared with loom.cpp, because three of the four are the engine's and this package inherits them by having no per-architecture code of its own.

1. GPUs and NPUs — the packaging is built; the backends beyond Vulkan are what remain. The engine schedules a graph across a device backend and a CPU fallback, this package exposes the choice as device= (above), and the wheel shape that lets an accelerator ship at all now exists.

The shape that is NOT wanted is a wheel per accelerator per architecture — PyPI's wheel tags have no accelerator dimension, so that is torch's cu121 arrangement, and it multiplies every future backend by every existing platform. GGML_BACKEND_DL makes the better shape possible and this package is now built that way: one arch-tagged base wheel, plus small backend packages that drop a .so where ggml looks for it, so pip install "loom-py-rt[cuda]" means "also fetch that backend", device="auto" finds it, and a Raspberry Pi installs nothing extra. packaging/rt-vulkan/ is the worked example; a CUDA package is that directory with two strings changed, waiting only on a machine with an NVIDIA GPU to build and test against. Tracked in Epic-04, along with which backends are reachable at all (CUDA, OpenVINO and Qualcomm's are already in the pinned ggml; CoreML and RKNPU2 are not).

2. Wheels for more platforms. Linux x86-64 and aarch64 and both Apple architectures today — see Supported platforms above, which is what pip install loom-py-rt resolves to. Windows is what remains, and it is behind the same kind of work macOS needed: not a missing CI row, but a symbol-visibility rule, a linker-path spelling and a build-system assumption that each differ (Epic-08, Retro-024).

3. More models — Epic-03, ordered by coverage per unit of effort. BERT token classifiers (the smallest possible template, and the first non-audio task), codec decoders, text encoder-decoders, CNN+CTC and SANM encoders are all done; what is left is the remaining TTS families → small classifiers → music. Each lands here for free: a model this package has never heard of works the day the exporter can produce it.

4. The follow-ups the docs already name — docs/backlog/active-index.md is the ledger for all three repos and the authority. The ones that would show up in this API: a permissively-licensed phonemiser, which is what would give the four phoneme-input TTS models a tokenize; and the KvCache memory redesign and quantized KV cache, which decide how large a model this can run on a given machine.

Licence

MIT — see LICENSE.