pylolm (LoLm)

Local LLMs on Android, from Python. An Android-friendly take on llama-cpp-python, built on llama.cpp and designed for Kivy, Flet and Termux.

• Zero Python dependencies: a thin ctypes layer over a small C shim.
• llama-cpp-python-style API: Llama(...), create_chat_completion(...), create_completion(...).
• Streaming generators, stop strings, a stop() button for your UI.
• Multi-turn chat that reuses the KV cache, so long conversations stay fast.
• Aimed at small GGUF models (0.5B-2B) on phones.

Status: alpha (0.1.0). Text generation only. TTS / STT / OCR are on the roadmap.

Install

pip install pylolm
pylolm get-libs      # downloads the prebuilt native libraries (seconds)
pylolm doctor        # checks that everything loads

get-libs fetches the Android arm64 libraries attached to the matching GitHub Release, checks their SHA-256 checksum, and installs them to ~/.pylolm/lib. These are built by CI to run on a wide range of phones.

Prefer to compile for your exact device, or get-libs is not available for your setup?

pkg install cmake clang git
pylolm build-native  # clones llama.cpp and builds on the device (slow: ~30 min)

Libraries are searched in $PYLOLM_LIB_DIR, then pylolm/lib/, then ~/.pylolm/lib/.

Quick start

from pylolm import Llama

llm = Llama("/sdcard/models/Qwen2.5-0.5B-Instruct-Q4_K_M.gguf", n_ctx=2048)

messages = [{"role": "user", "content": "Explain GGUF in two sentences."}]

# whole answer
out = llm.create_chat_completion(messages, max_tokens=200)
print(out["choices"][0]["message"]["content"])

# streaming
for chunk in llm.create_chat_completion(messages, max_tokens=200, stream=True):
    piece = chunk["choices"][0]["delta"].get("content")
    if piece:
        print(piece, end="", flush=True)

Try it from the terminal: pylolm chat /path/to/model.gguf.

API

Errors: LibraryNotFoundError, ModelLoadError, ContextOverflowError, GenerationError (all subclass PyLoLmError).

usage in a non-streaming result includes cached_prompt_tokens: how many prompt tokens were reused from the previous turn.

Chat templates. The model's built-in template is applied through llama.cpp's template support, which covers the common families (ChatML/Qwen, Llama 3, Gemma, ...) but is not a full Jinja engine. If a newer model's template is rejected, pass chat_template="chatml" (or another known name) or build the prompt yourself and call create_completion.

Using it in apps

UI threads. Generation blocks, so run it off the UI thread. A streaming generator is easiest:

import threading

def worker():
    for chunk in llm.create_chat_completion(messages, stream=True):
        text = chunk["choices"][0]["delta"].get("content")
        if text:
            Clock.schedule_once(lambda dt, t=text: append_to_label(t))  # Kivy
            # page.run_thread / page.update(...) for Flet

threading.Thread(target=worker, daemon=True).start()

Call llm.stop() from your Stop button; breaking out of the loop also stops generation.

Flet. flet build apk installs packages from PyPI using the Android platform tag, so the android_24_arm64_v8a wheel is what ships the native libraries. This path needs on-device verification for your Flet/Python version.

Kivy / python-for-android. p4a does not consume Android wheels yet. For now, bundle the .so files with your app and point PYLOLM_LIB_DIR at them before importing pylolm:

import os
os.environ["PYLOLM_LIB_DIR"] = os.path.join(os.path.dirname(__file__), "pylolm_libs")
import pylolm

A proper p4a recipe is on the roadmap.

Building from source

git clone https://github.com/auraway/pylolm && cd pylolm
pip install -e ".[dev]"
python -m pytest                                   # pure-Python tests, no model needed
PYLOLM_MODEL=/path/model.gguf python -m pytest     # + integration tests

Native libraries: src/pylolm/native/build_termux.sh (on a phone) or build_android_ndk.sh (cross-compile). The llama.cpp version is pinned in src/pylolm/native/LLAMA_CPP_REF.

Releasing to PyPI

1. Set LLAMA_CPP_REF to a llama.cpp tag/commit you have tested.
2. Bump version in pyproject.toml and add a CHANGELOG.md entry.
3. On PyPI, add a trusted publisher for this repo (workflow release.yml, environment pypi).
4. git tag v0.1.0 && git push --tags. CI builds the sdist, a pure wheel, and the Android arm64 wheel, publishes them to PyPI, and attaches pylolm-libs-android-arm64-v8a.tar.gz (+ .sha256) to the GitHub Release for pylolm get-libs. The tag must match the version in pyproject.toml.

Roadmap

• p4a recipe for Kivy; verified Flet build
• whisper.cpp (speech-to-text) module
• TTS and OCR modules
• GPU backends (Vulkan/OpenCL) as an opt-in

License

MIT. llama.cpp is MIT-licensed too; model licenses are your responsibility.