LocaLM

Run large language models on your own machine. Offline, private, and yours.

LocaLM downloads and runs LLMs locally: GGUF models through llama.cpp and HuggingFace models through transformers, on AMD, NVIDIA, Intel, Apple Silicon (experimental) or CPU. It ships a chat GUI, an OpenAI-compatible API, a coding agent, RAG over your own documents, and an MCP server, with every online feature off by default. The only thing it does online without being asked is a periodic update check (see Network below), which you can turn off too.

Install

pip install localm
localm setup-llama

setup-llama detects your GPU and provisions the matching llama.cpp runtime: NVIDIA gets a self-contained CUDA build (no CUDA Toolkit needed), AMD RX 6000 on Windows a bundled ROCm build, other AMD GPUs the ROCm/HIP build when a system ROCm/HIP toolkit is installed (Vulkan otherwise), Intel on Windows a self-contained SYCL build, Intel on Linux a Vulkan build, Apple Silicon Metal (experimental), and no GPU the CPU build. localm setup-llama --backend <name> picks one yourself.

Then pull a model and talk to it:

localm pull unsloth/Qwen3-4B-Instruct-2507-GGUF:Qwen3-4B-Instruct-2507-Q4_K_M.gguf
localm run Qwen3-4B-Instruct-2507-Q4_K_M

Or open the graphical interface:

localm gui

Python 3.12 is required (the package supports 3.12.x only, not 3.13 or newer). localm doctor reports what is installed and what is missing at any point.

Where data lives

Set LOCALM_HOME to choose where models, chats and settings are stored. Left unset, it defaults to a directory inside the Python environment you installed into (not a per-user directory); localm info prints the path actually in use.

Network

Online features (web access for the coder, cloud model providers) are opt-in. The one request LocaLM makes on its own is a periodic check for a new release, which sends no version number or chat content - turn it off with localm config net_mode off, or clear the update endpoint. Details: https://github.com/Matlan1/localm/blob/master/docs/privacy.md#4-update-checks-network-policy-not-a-persistence-mode

The self-contained installer

The pip package installs LocaLM into an environment you already manage. The installer on GitHub instead provisions its own Python and its own private environment, adds a desktop launcher and a native app window, and walks you through choosing plugins. If you would rather have that, or you are not on Python 3.12, use it:

https://github.com/Matlan1/localm

Documentation, issues and source

https://github.com/Matlan1/localm

AGPL-3.0-or-later.