Features

• Background server — start/stop/restart llama-server as a daemon
• Multi-model routing — preset-based configuration with automatic model load/unload
• Interactive downloads — search HuggingFace, pick a quant, download with progress and resume
• Rich terminal UI — tables, panels, interactive selectors, and live search
• GGUF inspector — view model metadata, architecture, and sampling parameters
• Server props — inspect active sampling parameters on loaded models
• Sampling sync — automatically applies GGUF-recommended sampling params to your preset
• Per-model settings — context size, GPU layers, flash attention, and more
• Multi-token prediction — models that ship an MTP head get speculative decoding switched on by themselves. A head shipped as a separate file is downloaded with the model and only wired in once the installed llama.cpp is confirmed to load it; the check runs again after every llama.cpp upgrade
• Idle sleeping — llama-server releases the weights of a model after a configurable idle timeout
• VRAM tracking — automatically parses server logs to show memory usage per model
• Auto-sync — preset file stays in sync with the llama.cpp cache automatically

Screenshots

Installation

pipx install llama-buddy

Or with uv:

uv tool install llama-buddy

This installs the llb command into an isolated environment and adds it to your PATH.

Prerequisites

• Python 3.10+
• llama.cpp installed and llama-server on your PATH

Quick start

# Download a model (interactive search)
llb download

# Or specify directly
llb download mistralai/Ministral-3-3B-Instruct-2512-GGUF:Q4_K_M

# Start the server
llb start

# List all models
llb models

# Chat with a model (uses llama-cli)
llb chat

# Inspect model metadata
llb info

# Show active sampling params for a loaded model
llb props

# Apply GGUF-recommended sampling params to all models
llb info --apply-sampling

# Configure settings (interactive TUI)
llb settings

# Open the web UI in your browser
llb open

# Stop the server
llb stop

Commands

Configuration

Config files live in ~/.config/llama/:

Per-model settings

Run llb settings and select Model Settings to configure per-model overrides:

• Context size, GPU layers, flash attention
• Custom aliases
• Any llama-server parameter

Development

# Clone and install
git clone https://github.com/thilomichael/llama-buddy.git
cd llama-buddy
uv sync

# Run
uv run llb <command>

# Test
uv run pytest

# Lint
uv run ruff check src/ tests/

License

MIT