loom-py

import loom

model = loom.Model.from_pretrained("loom-ai-org/lfm2-350m-monolithic-loom")
print(model.generate("The capital of France is", max_new_tokens=14))
# ':\nA) Paris\nB) Lyon\nC) Marseille\nD'

Text in, text out

generate tokenizes with the vocabulary the GGUF embeds, runs the driver, and detokenizes what comes back. The same steps are available separately when you want them:

model.tokenize("The capital of France is")   # [1, 1098, 5706, 803, 4481, 856]
model.detokenize([1, 1098, 5706])            # '<|startoftext|>The capital'
model.tokenizer                               # <loom.Tokenizer 'gpt2' size=64400>

The four vocabulary families a loom GGUF can carry — byte-level BPE, SentencePiece, WordPiece and byte-level — are dispatched on the file's own tokenizer.ggml.model, so this is one call whichever one a model uses.

For a speech model there is nothing to encode; detokenizing the driver's output is the other half of the same thing:

transcript = model.detokenize(model.infer(waveform=audio, audio_samples=len(audio)))

A TTS model has no generate, and that is a real limitation rather than a missing feature. Matcha, VITS, Kokoro and StyleTTS2 consume phoneme ids that a phonemiser produces outside the engine, so their GGUFs embed no vocabulary at all — model.tokenizer is None for them and they take ids directly:

audio = model.infer(tokens=[16, 40, 22, 30, 12, 3], n_steps=4, seed=1234)

Choosing a device

A wheel built with a GPU backend uses it by default; one built without has only a CPU to find, so nothing changes.

model = loom.Model.from_file("qwen3.gguf")                  # decide for me (or $LOOM_DEVICE)
model = loom.Model.from_file("qwen3.gguf", device="cpu")    # pin it
model = loom.Model.from_file("qwen3.gguf", device="gpu")    # demand one; raises if there is none
model.device, model.device_description   # ('Vulkan0', 'AMD Radeon Vega 3 Graphics (RADV RAVEN2)')

"gpu" raises rather than falling back, because a caller who spelled it out is asking a question about the machine and a silent CPU run is how a large slowdown goes unnoticed. "auto" — the default — is the one that falls back.

The wheels on PyPI are CPU-only, so a GPU today means building from a checkout with the engine configured for one (CMAKE_ARGS="-DGGML_VULKAN=ON" pip install -e .); see loom.cpp's own build notes. Which ops fall back to the CPU, and why some always will, is documented there too.

Why there is so little API

A loom GGUF carries its own graph topologies and its own driver script alongside its weights, so this package contains no per-architecture code at all. Loading a model registers whatever topologies the file declares and attaches a KV cache to the ones that say they need it; running one calls the driver the file shipped with. A model this library has never heard of works the day loom-exporter can produce it.

That is also why infer takes **kwargs: its arguments are the driver's arguments, and which ones a model takes is a property of the model. model.driver_source prints the Lua that will run, whose header comment documents its inputs — that is the authority.

model = loom.Model.from_file("granite_speech_mil.gguf")
model.architecture          # 'granite-speech'
model.topologies            # ['encoder', 'embed', 'decoder', 'lm_head']
model.hparam("samples_per_chunk")   # 192000
print(model.driver_source)  # what infer() will run, and what it accepts

Supported models

Seventeen, published at huggingface.co/loom-ai-org and loadable by id with from_pretrained (needs the [hub] extra). This package has no per-architecture code, so the list is a property of loom-exporter, not of anything here.

Language models

These are the models generate works on.

Speech recognition

model.detokenize(model.infer(waveform=audio, audio_samples=len(audio))) — the mel frontend is inside the graph, so a raw waveform is the input.

Speech synthesis

Supertonic is the one with a text door — model.tokenize works on it and is None for the other four, which take phoneme ids (see above). model.driver_source is the authority on what each accepts.

The three repos

Installing

pip install loom-py-rt           # once published -- `loom-py` on PyPI clashes with `loompy`,
                                  # and `loom-engine` normalizes to the already-taken `loomengine`
pip install loom-py-rt[hub]      # + from_pretrained()

From a checkout — note --recursive, since the engine is a submodule:

git clone --recursive https://github.com/loom-ai-org/loom-py
cd loom-py && pip install -e .

No runtime dependencies. Arrays cross the boundary as plain sequences of floats, so numpy is something you may use rather than something this package makes you install — list, array.array, numpy arrays and torch tensors all work.

Testing

pytest tests/ci      # the Python layer: coercion and error paths. No model. What CI runs.
pytest tests/gate    # a real exported GGUF, end to end.

export LOOM_TEST_MODEL=~/loom-fixtures/matcha_mil.gguf
export LOOM_TEST_MODEL_INPUTS='{"tokens":[16,40,22,30,12,3],"n_steps":4,"seed":1234}'
pytest tests/gate -q

The gate suite is written against no particular architecture on purpose: it asserts the shape of what a loom model is, which is the whole of what this package knows. A test that expected one model's inputs would be this package learning about a model, which is the thing the design exists to avoid.

Roadmap

Shared with loom.cpp, because three of the four are the engine's and this package inherits them by having no per-architecture code of its own.

1. GPUs and NPUs — the engine part is done; the packaging part is the open question. The engine schedules a graph across a device backend and a CPU fallback, and this package exposes the choice as device= (above). What is missing from here is a wheel that has a device backend in it: today a GPU means building from a checkout.

The shape that is NOT wanted is a wheel per accelerator per architecture — PyPI's wheel tags have no accelerator dimension, so that is torch's cu121 arrangement, and it multiplies every future backend by every existing platform. GGML_BACKEND_DL (verified on the engine side) makes the better shape possible: one arch-tagged base wheel, plus small backend packages that drop a .so where ggml looks for it, so pip install loom-py-rt[cuda] means "also fetch that backend", device="auto" finds it, and a Raspberry Pi installs nothing extra. Scoped as BACKLOG.md P4.8, along with which backends are reachable at all (CUDA, OpenVINO and Qualcomm's are already in the pinned ggml; CoreML and RKNPU2 are not).

2. Wheels for more platforms. Linux x86-64 today; next macOS on Intel, macOS on Apple Silicon and Linux on ARM. This is the item most visible from here, since it is what pip install loom-py-rt can resolve to.

3. More models — P5 in the ledger, ordered by coverage per unit of effort: BERT token classifiers (the smallest possible template, and the first non-audio task) → codec decoders → CNN+CTC and SANM encoders → the remaining TTS families → text encoder-decoders → small classifiers → music. Each lands here for free: a model this package has never heard of works the day the exporter can produce it.

4. The follow-ups the docs already name — BACKLOG.md is the ledger for all three repos and the authority. The ones that would show up in this API: a permissively-licensed phonemiser, which is what would give the four phoneme-input TTS models a tokenize; and the KvCache memory redesign and quantized KV cache, which decide how large a model this can run on a given machine.

Licence

MIT — see LICENSE.