loom-py

import loom

model = loom.Model.from_pretrained("loom-ai-org/lfm2-350m-monolithic-loom")
print(model.generate("The capital of France is", max_new_tokens=14))
# ':\nA) Paris\nB) Lyon\nC) Marseille\nD'

Text in, text out

generate tokenizes with the vocabulary the GGUF embeds, runs the driver, and detokenizes what comes back. The same steps are available separately when you want them:

model.tokenize("The capital of France is")   # [1, 1098, 5706, 803, 4481, 856]
model.detokenize([1, 1098, 5706])            # '<|startoftext|>The capital'
model.tokenizer                               # <loom.Tokenizer 'gpt2' size=64400>

The four vocabulary families a loom GGUF can carry — byte-level BPE, SentencePiece, WordPiece and byte-level — are dispatched on the file's own tokenizer.ggml.model, so this is one call whichever one a model uses.

For a speech model there is nothing to encode; detokenizing the driver's output is the other half of the same thing:

transcript = model.detokenize(model.infer(waveform=audio, audio_samples=len(audio)))

A TTS model has no generate, and that is a real limitation rather than a missing feature. Matcha, VITS, Kokoro and StyleTTS2 consume phoneme ids that a phonemiser produces outside the engine, so their GGUFs embed no vocabulary at all — model.tokenizer is None for them and they take ids directly:

audio = model.infer(tokens=[16, 40, 22, 30, 12, 3], n_steps=4, seed=1234)

Choosing a device

A wheel built with a GPU backend uses it by default; one built without has only a CPU to find, so nothing changes.

model = loom.Model.from_file("qwen3.gguf")                  # decide for me (or $LOOM_DEVICE)
model = loom.Model.from_file("qwen3.gguf", device="cpu")    # pin it
model = loom.Model.from_file("qwen3.gguf", device="gpu")    # demand one; raises if there is none
model.device, model.device_description   # ('Vulkan0', 'AMD Radeon Vega 3 Graphics (RADV RAVEN2)')

"gpu" raises rather than falling back, because a caller who spelled it out is asking a question about the machine and a silent CPU run is how a large slowdown goes unnoticed. "auto" — the default — is the one that falls back.

The base wheel is CPU-only, and an accelerator is a separate install rather than a different wheel:

pip install "loom-py-rt[vulkan]"

That adds one small package holding one libggml-vulkan.so, which this package finds at import; device="auto" then uses it and nothing about the base wheel changes. The reason it works this way — rather than a full wheel per accelerator, which is the more familiar shape — is that a Vulkan backend is 46.5 MB and CUDA is larger, so the per-accelerator matrix does not fit PyPI's 100 MB per-file ceiling. See packaging/README.md.

loom.devices()   # [{'name': 'Vulkan0', 'description': 'AMD Radeon Vega 3 Graphics (RADV RAVEN2)', ...}]

Worth calling after installing one, because a backend whose driver is too old — or which finds no supported device — loads without error and registers nothing, and the only other symptom is a model running at CPU speed. Note that with this build every backend is loaded at run time, the CPU included, so an empty device list means no backend library was found at all rather than no accelerator.

Which ops fall back to the CPU, and why some always will, is documented in loom.cpp's own build notes.

Why there is so little API

A loom GGUF carries its own graph topologies and its own driver script alongside its weights, so this package contains no per-architecture code at all. Loading a model registers whatever topologies the file declares and attaches a KV cache to the ones that say they need it; running one calls the driver the file shipped with. A model this library has never heard of works the day loom-exporter can produce it.

That is also why infer takes **kwargs: its arguments are the driver's arguments, and which ones a model takes is a property of the model. model.driver_source prints the Lua that will run, whose header comment documents its inputs — that is the authority.

model = loom.Model.from_file("granite_speech_mil.gguf")
model.architecture          # 'granite-speech'
model.topologies            # ['encoder', 'embed', 'decoder', 'lm_head']
model.hparam("samples_per_chunk")   # 192000
print(model.driver_source)  # what infer() will run, and what it accepts

Supported models

Seventeen, published at huggingface.co/loom-ai-org and loadable by id with from_pretrained (needs the [hub] extra). This package has no per-architecture code, so the list is a property of loom-exporter, not of anything here.

Language models

These are the models generate works on.

Speech recognition

model.detokenize(model.infer(waveform=audio, audio_samples=len(audio))) — the mel frontend is inside the graph, so a raw waveform is the input.

Speech synthesis

Supertonic is the one with a text door — model.tokenize works on it and is None for the other four, which take phoneme ids (see above). model.driver_source is the authority on what each accepts.

The three repos

Installing

pip install loom-py-rt           # once published -- `loom-py` on PyPI clashes with `loompy`,
                                  # and `loom-engine` normalizes to the already-taken `loomengine`
pip install loom-py-rt[hub]      # + from_pretrained()

From a checkout — note --recursive, since the engine is a submodule:

git clone --recursive https://github.com/loom-ai-org/loom-py
cd loom-py && pip install -e .

No runtime dependencies. Arrays cross the boundary as plain sequences of floats, so numpy is something you may use rather than something this package makes you install — list, array.array, numpy arrays and torch tensors all work.

Testing

pytest tests/ci      # the Python layer: coercion and error paths. No model. What CI runs.
pytest tests/gate    # a real exported GGUF, end to end.

export LOOM_TEST_MODEL=~/loom-fixtures/matcha_mil.gguf
export LOOM_TEST_MODEL_INPUTS='{"tokens":[16,40,22,30,12,3],"n_steps":4,"seed":1234}'
pytest tests/gate -q

The gate suite is written against no particular architecture on purpose: it asserts the shape of what a loom model is, which is the whole of what this package knows. A test that expected one model's inputs would be this package learning about a model, which is the thing the design exists to avoid.

Roadmap

Shared with loom.cpp, because three of the four are the engine's and this package inherits them by having no per-architecture code of its own.

1. GPUs and NPUs — the packaging is built; the backends beyond Vulkan are what remain. The engine schedules a graph across a device backend and a CPU fallback, this package exposes the choice as device= (above), and the wheel shape that lets an accelerator ship at all now exists.

The shape that is NOT wanted is a wheel per accelerator per architecture — PyPI's wheel tags have no accelerator dimension, so that is torch's cu121 arrangement, and it multiplies every future backend by every existing platform. GGML_BACKEND_DL makes the better shape possible and this package is now built that way: one arch-tagged base wheel, plus small backend packages that drop a .so where ggml looks for it, so pip install "loom-py-rt[cuda]" means "also fetch that backend", device="auto" finds it, and a Raspberry Pi installs nothing extra. packaging/rt-vulkan/ is the worked example; a CUDA package is that directory with two strings changed, waiting only on a machine with an NVIDIA GPU to build and test against. Tracked as BACKLOG.md P4.8, along with which backends are reachable at all (CUDA, OpenVINO and Qualcomm's are already in the pinned ggml; CoreML and RKNPU2 are not).

2. Wheels for more platforms. Linux x86-64 today; next macOS on Intel, macOS on Apple Silicon and Linux on ARM. This is the item most visible from here, since it is what pip install loom-py-rt can resolve to.

3. More models — P5 in the ledger, ordered by coverage per unit of effort: BERT token classifiers (the smallest possible template, and the first non-audio task) → codec decoders → CNN+CTC and SANM encoders → the remaining TTS families → text encoder-decoders → small classifiers → music. Each lands here for free: a model this package has never heard of works the day the exporter can produce it.

4. The follow-ups the docs already name — BACKLOG.md is the ledger for all three repos and the authority. The ones that would show up in this API: a permissively-licensed phonemiser, which is what would give the four phoneme-input TTS models a tokenize; and the KvCache memory redesign and quantized KV cache, which decide how large a model this can run on a given machine.

Licence

MIT — see LICENSE.