lalamo
A set of tools for adapting Large Language Models to on-device inference using the uzu inference engine.
Quick Start
To get the list of supported models, run:
uv run lalamo list-models
To convert a model, run:
uv run lalamo convert MODEL_REPO
Note: on some CPU platform you may be getting an error saying The precision 'F16_F16_F32' is not supported by dot_general on CPU. This is due to a bug in XLA, which causes matmuls inside jax.jit not work correctly on CPUs. The workaround is to set the environment variable JAX_DISABLE_JIT=1 when running the conversion.
After that, you can find the converted model in the models folder. For more options see uv run lalamo convert --help.
Model Support
To add support for a new model, write the corresponding ModelSpec, as shown in the example below:
ModelSpec(
vendor="Google",
family="Gemma-3",
name="Gemma-3-1B-Instruct",
size="1B",
quantization=None,
repo="google/gemma-3-1b-it",
config_type=HFGemma3TextConfig,
weights_type=WeightsType.SAFETENSORS,
)
Optional Features
PyAudio
PyAudio enables audio playback for TTS models and is used as an optional Lalamo feature because it requires PortAudio to be installed.
How to:
- macOS:
brew install portaudio(formula) - Debian/Ubuntu:
apt-get install portaudio19-dev python-all-dev - Other Linux: PortAudio build instructions
Then run :
uv run --with pyaudio lalamo path/to/model --replay
1 maintainer
Release files for lalamo 0.16.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| lalamo-0.16.0.tar.gz | 376.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| lalamo-0.16.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 841.2 kB
Release files / lalamo-0.16.0.tar.gz
| Download URL | lalamo-0.16.0.tar.gz |
|---|---|
| Size | 376.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
1bb2d3fba20829564f6b1c033f7d1a0fea0214e8c096ebd3b36bd8ebb598b8d8
|
|
BLAKE2b-256 checksum How to use checksums |
085319ed7f0e13e213014918e47d6fb414fc297873b74e59af07be4d2687e2f8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.8 {"installer":{"name":"uv","version":"0.12.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / lalamo-0.16.0-py3-none-any.whl
| Download URL | lalamo-0.16.0-py3-none-any.whl |
|---|---|
| Size | 464.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a46a06a4f2aa361df5f9706119fe7dd7d78b8538d8e97c9db8e4e3e038cba1eb
|
|
BLAKE2b-256 checksum How to use checksums |
26c5436f6afda81c1eb507931e0f1e04ac0c5dd5bdc58ef42682a887d8a88e92
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.8 {"installer":{"name":"uv","version":"0.12.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
