lalamo
A set of tools for adapting Large Language Models to on-device inference using the uzu inference engine.
Quick Start
To get the list of supported models, run:
uv run lalamo list-models
To convert a model, run:
uv run lalamo convert MODEL_REPO
Note: on some CPU platform you may be getting an error saying The precision 'F16_F16_F32' is not supported by dot_general on CPU. This is due to a bug in XLA, which causes matmuls inside jax.jit not work correctly on CPUs. The workaround is to set the environment variable JAX_DISABLE_JIT=1 when running the conversion.
After that, you can find the converted model in the models folder. For more options see uv run lalamo convert --help.
Model Support
To add support for a new model, write the corresponding ModelSpec, as shown in the example below:
ModelSpec(
vendor="Google",
family="Gemma-3",
name="Gemma-3-1B-Instruct",
size="1B",
quantization=None,
repo="google/gemma-3-1b-it",
config_type=HFGemma3TextConfig,
weights_type=WeightsType.SAFETENSORS,
)
Optional Features
PyAudio
PyAudio enables audio playback for TTS models and is used as an optional Lalamo feature because it requires PortAudio to be installed.
How to:
- macOS:
brew install portaudio(formula) - Debian/Ubuntu:
apt-get install portaudio19-dev python-all-dev - Other Linux: PortAudio build instructions
Then run :
uv run --with pyaudio lalamo path/to/model --replay
1 maintainer
Release files for lalamo 0.17.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| lalamo-0.17.0.tar.gz | 379.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| lalamo-0.17.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 846.4 kB
Release files / lalamo-0.17.0.tar.gz
| Download URL | lalamo-0.17.0.tar.gz |
|---|---|
| Size | 379.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6334c110827f2adb8c8c4df1c8505303a9cf10f394c12efd5a6f396664b7d4a1
|
|
BLAKE2b-256 checksum How to use checksums |
b1aaf70806bbd455374ea688316af018d288215f3325d2532a3bb5d9ee4c8e11
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.18 {"installer":{"name":"uv","version":"0.12.18","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / lalamo-0.17.0-py3-none-any.whl
| Download URL | lalamo-0.17.0-py3-none-any.whl |
|---|---|
| Size | 467.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ce16fd1d6f3488e81ef8e8ebcf8a60ca7e75f445877b5f8e4f3189b2356423d9
|
|
BLAKE2b-256 checksum How to use checksums |
531c01712d0dc7698ff74e680e925ff777d3c068090f4d3c3761fbffa2436a4d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.18 {"installer":{"name":"uv","version":"0.12.18","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
