geistr

Run language models on your own computer: download, chat, serve. One small binary, no Python, no Docker, no account. The model runs locally, on the CPU or the GPU.

Install

curl -fsSL https://www.geisten.net/download/geistr/install.sh | sh

Or with Homebrew:

brew install geisten/tap/geistr

Both work on Linux (x86_64, arm64) and macOS (Apple Silicon). The script downloads the latest release, checks it against SHA256SUMS and puts geistr in /usr/local/bin.

Without sudo, or a specific release (a release that is not on the server is reported as "no release …"; releases before 0.3.2 never were):

curl -fsSL https://www.geisten.net/download/geistr/install.sh | PREFIX=~/.local sh
curl -fsSL https://www.geisten.net/download/geistr/install.sh | GEISTR_VERSION=v0.3.2 sh

Quick start

geistr chat                    # the first time: suggests a model for this computer,
                               # downloads and verifies it on Enter, then chats

Or choose yourself:

geistr catalog                 # which models fit this computer
geistr pull qwen3-0.6b         # download and verify one (640 MB)
geistr chat qwen3-0.6b         # talk to it

Or a single answer, for scripts:

geistr run qwen3-0.6b "Name three prime numbers."
echo "Summarize: …" | geistr run qwen3-0.6b

Models

geistr catalog lists the models geistr knows, the best suited to this computer first (installed or not does not move a row), marks what is installed (✓), what is available (↓) and whether each fits your RAM (⚠/✗), with the speed measured here on ⚙ CPU and ⚡ GPU.

Every download is checked against its SHA-256 and resumes where it stopped. Any other .gguf file works too: geistr chat path/to/model.gguf.

Chat

In a terminal, geistr chat renders Markdown, tables, math and clickable links, shows the speed after each answer and continues your last conversation (see below). Once the context is half full, the prompt says how full (⚙ 62% >, yellow from 80 %, red from 95 %); /info always shows it. Type / for the commands:

/model gemma4-e2b    switch the model, keep the conversation (best fit first)
/gpu /cpu /auto      switch the processor (Metal on macOS, Vulkan on Linux)
/temp 0.7            sampling temperature
/system Be brief.    system prompt (/system off removes it)
/info                what runs now, and how full the context is
/files ~/Notes       answers from your files (/files off: none)
/retry               the last answer again (at temperature 0: once at 0.7)
/copy  /copy code    the last answer, or its last code block, to the clipboard
/save                keep model, processor, temperature and system prompt
/clear  /exit

Your conversation stays

Close the terminal, reboot, come back tomorrow: geistr chat picks up where you left off, and the model still knows what you talked about.

↻ 6 · „Compare Mars and Venus in a small table…“ · /clear new
⚡ >

That is what makes a local model useful for real work: you can think a problem through over several sessions, refer back to an earlier answer, or switch to a bigger model mid-conversation without explaining everything again. It is also private by design: each conversation is a plain file on your computer (readable only by you, one JSON message per line), never sent anywhere.

A long conversation stays fast to resume: the model re-reads only the newest part (about 2000 tokens, geistr config resume_tokens), and the ↻ line says how much. The whole conversation stays in the file. When a conversation outgrows the model's context, the oldest messages are left out, and the chat says so (↥ 12 oldest messages left out …).

• /clear starts a new conversation and keeps the old file.
• geistr chat --new starts fresh once; geistr config resume off always.
• Piped chats (echo … | geistr chat) keep nothing, so scripts stay reproducible.

Ask your own files

geistr pull bitnet-embed-0.6b      # the embedding model, 428 MB, once
geistr index ~/Notes               # text, Markdown, PDF (with pdftotext)
geistr chat --files ~/Notes        # or /files ~/Notes in a chat

Each question gets the most relevant passages of your files put in front of it, and the answer shows where they came from (sources: garden/shed.md). geistr run --files ~/Notes "…" does the same once. The index is updated by file hash whenever you use the folder, so only new and changed files are read again. It stays on your computer, readable only by you, in the data folder (index/); hidden files and folders are left out.

Use it from other tools

Keep a model loaded and talk to it over the OpenAI or Ollama API, e.g. from Open WebUI, editor plugins or the OpenAI SDK:

geistr serve gemma4-e2b --http          # http://127.0.0.1:11434

curl http://127.0.0.1:11434/v1/chat/completions \
  -d '{"messages":[{"role":"user","content":"Hi"}]}'

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="unused")
reply = client.chat.completions.create(model="gemma4-e2b",
                                       messages=[{"role": "user", "content": "Hi"}])
print(reply.choices[0].message.content)

In Open WebUI, add http://127.0.0.1:11434 as an Ollama connection (or …/v1 as OpenAI). Clients that resend the whole conversation pay only for what is new.

echo '{"op":"chat","messages":[{"role":"user","content":"Hi"}]}' | nc -U ~/…/geistr.sock

Settings

Settings are remembered between runs; geistr config shows them and the file they live in.

geistr config processor gpu                # auto (default), cpu, gpu
geistr config temperature 0.7              # 0 to 2
geistr config system "Answer in German."   # system prompt for every new chat
geistr config markdown off                 # plain text
geistr config stats off                    # no speed line
geistr config resume off                   # always start new, keep nothing
geistr config history off                  # don't keep typed lines between chats
geistr config threads 8                    # CPU threads (default: one per physical core)

--cpu / --gpu choose the processor for one run, --threads N the CPU threads, --models DIR another model folder.

Update

brew upgrade geistr, or run the install command again. The model list ships inside geistr, so new models come with a new geistr. Afterwards geistr catalog marks installed models whose file changed with ⟳ and a file of the right size with another checksum (damaged?) with ✗, and geistr pull fetches them (the old file stays until the new one is verified).

Python and C

geistr is built on geist-runtime, an embeddable model runner on geistlib with one C API (include/geistr.h) and a Python package (pip install geistr; every release also has its wheels at https://www.geisten.net/download/geistr/vX.Y.Z/):

import geistr

with geistr.chat("gemma4-e2b", system="Answer briefly.") as chat:
    for piece in chat.send("What is the capital of France?"):   # streamed
        print(piece, end="", flush=True)
    print(chat.ask("And of Italy?"))           # only the new message is processed

Building, the library, tests and CI: docs/DEVELOPMENT.md. API reference and design: docs/API.md.