llama-tui 0.1.2

Lightweight TUI manager for llama.cpp models on Raspberry Pi

llama-tui

CLI manager for llama.cpp. Detects hardware, downloads models, launches optimized servers. Specifically made for Raspberry Pi OS (Linux).

Requirements

• Raspberry Pi OS (any model, 64-bit recommended)
• Python 3.11+
• uv
• llama-server (compiled with BLIS + NEON)
• hf CLI (huggingface-hub, authenticated for private models)

Install

uv tool install llama-tui

Setup

llama-tui setup    # detect hardware, select models
llama-tui download # fetch enabled models
llama-tui prep     # system tuning (swap, governor, hugepages)

Notes:

• llama-tui prep and llama-tui serve use sudo for system tuning. Run from a terminal that can prompt for sudo.
• Config is stored at $XDG_CONFIG_HOME/llama-tui/config.toml (defaults to ~/.config/llama-tui/config.toml).
• Config models are stored as [[models]] array-of-tables entries.

Serve

llama-tui serve --role chat

/health returns 503 while the model is loading and 200 when ready; llama-tui waits for 200 before returning.

Other Commands

llama-tui models   # list enabled models
llama-tui rpc      # check remote RPC connectivity
llama-tui status   # full system & config summary
llama-tui restore  # revert system tuning
llama-tui swap     # show swap status

Optional RPC

On a second Pi, run:

rpc-server --host <hostname> --port 50052

Enable RPC during setup or edit ~/.config/llama-tui/config.toml.

License

MIT See LICENSE.