lalamo 0.2.4

JAX library for optimization and export of models for use with the UZU inference engine.

lalamo

A set of tools for adapting Large Language Models to on-device inference using the uzu inference engine.

Quick Start

To get the list of supported models, run:

uv run lalamo list-models

To convert a model, run:

uv run lalamo convert MODEL_REPO --precision float16

After that, you can find the converted model in the models folder. For more options see uv run lalamo convert --help.

Model Support

To add support for a new model, write the corresponding ModelSpec, as shown in the example below:

ModelSpec(
    vendor="Google",
    family="Gemma-3",
    name="Gemma-3-1B-Instruct",
    size="1B",
    quantization=None,
    repo="google/gemma-3-1b-it",
    config_type=HFGemma3TextConfig,
    config_file_name="config.json",
    weights_file_names=huggingface_weight_files(1),
    weights_type=WeightsType.SAFETENSORS,
    tokenizer_files=HUGGINGFACE_TOKENIZER_FILES,
    use_cases=tuple(),
)