geistr

Run language models on your own computer: download, chat, serve. One small binary, no Python, no Docker, no account. The model runs locally, on the CPU or the GPU.

Full manual: docs/GUIDE.md: every command, chat command, setting and file, the HTTP API, and what to do when something fails.

Install

curl -fsSL https://www.geisten.net/download/geistr/install.sh | sh

Or with Homebrew:

brew install geisten/tap/geistr

Both work on Linux (x86_64, arm64) and macOS (Apple Silicon). The script downloads the latest release, checks it against SHA256SUMS and puts geistr in /usr/local/bin.

Without sudo, or a specific release (the server keeps the last 5, from 0.3.2 on; another one is reported as "no release …"):

curl -fsSL https://www.geisten.net/download/geistr/install.sh | PREFIX=~/.local sh
curl -fsSL https://www.geisten.net/download/geistr/install.sh | GEISTR_VERSION=v0.3.4 sh

Requirements

The Linux binaries are fully static and run on any distribution. The CPU needs x86-64-v3 (AVX2: Intel since 2013, AMD since 2015) or ARMv8.2 with dotprod (Raspberry Pi 5, AWS Graviton 2 and newer); on an older CPU geistr says so and exits. To build from source instead, see docs/DEVELOPMENT.md.

Quick start

geistr chat                    # the first time: suggests a model for this computer,
                               # downloads and verifies it on Enter, then chats

Or choose yourself:

geistr catalog                 # which models fit this computer
geistr pull qwen3-0.6b         # download and verify one (640 MB)
geistr chat qwen3-0.6b         # talk to it

Or a single answer, for scripts:

geistr run qwen3-0.6b "Name three prime numbers."
echo "Summarize: …" | geistr run qwen3-0.6b

Models

geistr catalog lists the models geistr knows, the best suited to this computer first (installed or not does not move a row), marks what is installed (✓), what is available (↓) and whether each fits your RAM (⚠/✗), with the speed measured here on ⚙ CPU and ⚡ GPU.

model	download	RAM
smollm2-360m	0.4 GB	2 GB
qwen3-0.6b	0.6 GB	4 GB
qwen35-0.8b	0.8 GB	4 GB
bitnet-2b	1.2 GB	4 GB
gemma4-e2b	3.1 GB	8 GB
gemma4-e4b	5.0 GB	16 GB
bonsai2-27b-pq2	7.2 GB	24 GB
qwen38-27b-q4	16 GB	32 GB
qwen38-27b-q8	29 GB	48 GB

Every download is checked against its SHA-256 and resumes where it stopped. Any other .gguf file works too: geistr chat path/to/model.gguf.

Chat

In a terminal, geistr chat renders Markdown, tables, math and clickable links, shows the speed after each answer and continues your last conversation (see below). Once the context is half full, the prompt says how full (⚙ 62% >, yellow from 80 %, red from 95 %); /info always shows it. Type / for the commands:

/model gemma4-e2b    switch the model, keep the conversation (best fit first)
/gpu /cpu /auto      switch the processor (Metal on macOS, Vulkan on Linux)
/temp 0.7            sampling temperature
/system Be brief.    system prompt (/system off removes it)
/info                what runs now, and how full the context is
/files ~/Notes       answers from your files (/files off: none)
/retry               the last answer again (at temperature 0: once at 0.7)
/copy  /copy code    the last answer, or its last code block, to the clipboard
/save                keep model, processor, temperature and system prompt
/clear  /exit

key	does
Ctrl-J, Alt-Enter, \ + Enter	a new line in the message (Enter sends)
Esc	stop the answer
Ctrl-C	clear the line; twice on an empty line: exit
↑ ↓	earlier lines, also from earlier chats; → takes the hint
Ctrl-R	search earlier lines (type to narrow, Ctrl-R for older, Enter takes it)
?	the shortcuts

Your conversation stays

Close the terminal, reboot, come back tomorrow: geistr chat picks up where you left off, and the model still knows what you talked about.

↻ 6 · „Compare Mars and Venus in a small table…“ · /clear new
⚡ >

That is what makes a local model useful for real work: you can think a problem through over several sessions, refer back to an earlier answer, or switch to a bigger model mid-conversation without explaining everything again. It is also private by design: each conversation is a plain file on your computer (readable only by you, one JSON message per line), never sent anywhere.

A long conversation stays fast to resume: the model re-reads only the newest part (about 2000 tokens, geistr config resume_tokens), and the ↻ line says how much. The whole conversation stays in the file. When a conversation outgrows the model's context, the oldest messages are left out, and the chat says so (↥ 12 oldest messages left out …).

• /clear starts a new conversation and keeps the old file.
• geistr chat --new starts fresh once; geistr config resume off always.
• Piped chats (echo … | geistr chat) keep nothing, so scripts stay reproducible.

Ask your own files

geistr pull bitnet-embed-0.6b      # the embedding model, 428 MB, once
geistr index ~/Notes               # text, Markdown, PDF (with pdftotext)
geistr chat --files ~/Notes        # or /files ~/Notes in a chat

Each question gets the most relevant passages of your files put in front of it, and the answer shows where they came from (sources: garden/shed.md). With the reranker installed (geistr pull qwen3-reranker-0.6b, 639 MB) only the passages that answer the question go in front of it and only their files are named; for a question your files do not answer it says nothing relevant in your files. Without it, the four nearest passages are used and named. geistr run --files ~/Notes "…" does the same once. The index is updated by file hash whenever you use the folder, so only new and changed files are read again. It stays on your computer, readable only by you, in the data folder (index/); hidden files and folders are left out.

Use it from other tools

Keep a model loaded and talk to it over the OpenAI or Ollama API, e.g. from Open WebUI, editor plugins or the OpenAI SDK:

geistr serve gemma4-e2b --http          # http://127.0.0.1:11434

curl http://127.0.0.1:11434/v1/chat/completions \
  -d '{"messages":[{"role":"user","content":"Hi"}]}'

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="unused")
reply = client.chat.completions.create(model="gemma4-e2b",
                                       messages=[{"role": "user", "content": "Hi"}])
print(reply.choices[0].message.content)

In Open WebUI, add http://127.0.0.1:11434 as an Ollama connection (or …/v1 as OpenAI). Clients that resend the whole conversation pay only for what is new.

Endpoints, the socket, and security

endpoint
POST /v1/chat/completions	OpenAI: messages, temperature, max_tokens / max_completion_tokens, stop, stream (SSE, stream_options.include_usage), tools / tool_choice
POST /v1/embeddings	OpenAI: input (a text or a list), encoding_format (float, base64); with an embedding model
GET /v1/models	the one model
POST /api/chat	Ollama: messages, stream (NDJSON, default), options.temperature, num_predict, stop, tools
POST /api/embed	Ollama: input (a text or a list); with an embedding model
GET /api/tags, /api/version, /	the model, the version, a health check

There is no authentication. By default geistr listens on loopback only and answers only requests addressed to this computer (Host check, against DNS rebinding). A web page in a browser may use it only if it comes from this computer too (Origin check: another site's page cannot start answers); GEISTR_ORIGINS=https://ui.example,… (or *) allows more, like Ollama's OLLAMA_ORIGINS. --http=0.0.0.0:11434 makes it reachable for everyone who can reach the computer; geistr warns about that. One request is answered at a time; a client that disconnects stops its answer.

Tool calling works with Qwen3 models (e.g. qwen3-0.6b): send tools, get tool_calls back (finish_reason: tool_calls; streamed too), send the results as role: tool messages, and the model answers with them, as in the OpenAI SDK's function-calling example. tool_choice: "none" turns tools off for a request. A model without a tool format refuses tools (400).

Images work with Gemma 4 E2B (gemma4-e2b): geistr pull gemma4-e2b also fetches its vision tower (the vision tensors of the original model, a 337 MB vision_tower.safetensors next to it, SHA-256 verified). Send an image_url part with a data: URL (OpenAI) or images (Ollama), PNG, JPEG or BMP, one per message; the image goes with the last message, and the conversation keeps it for the follow-up questions. Other models refuse images (400).

Embeddings: geistr serve bitnet-embed-0.6b --http answers /v1/embeddings (the OpenAI SDK's embeddings.create works) and Ollama's /api/embed: unit-length vectors of 1024 dimensions, multilingual. For retrieval, embed the question and the documents as they are, with nothing before them, as --files does: on scripts/retrieval-bench.py (#157) that put the right file among the four nearest passages most often (97 %), ahead of an Instruct: …\nQuery: task before the question (92 %) or query: / passage: (89–94 %). An embedding model does not chat, and a chat model does not embed (400). Not supported yet: more than one model per service.

Without --http, geistr serve listens on a Unix socket only (owner only, 0600), and geistr chat --socket chats through it. --chats N sets how many conversations the service keeps (default 2), --max-request=MiB the size of one request (default 16). The options and the socket protocol: docs/GUIDE.md.

Settings

Settings are remembered between runs; geistr config shows them and the file they live in.

geistr config processor gpu                # auto (default), cpu, gpu
geistr config temperature 0.7              # 0 to 2
geistr config system "Answer in German."   # system prompt for every new chat
geistr config markdown off                 # plain text
geistr config stats off                    # no speed line
geistr config resume off                   # always start new, keep nothing
geistr config history off                  # don't keep typed lines between chats
geistr config threads 8                    # CPU threads (default: one per physical core)

--cpu / --gpu choose the processor for one run, --threads N the CPU threads, --models DIR another model folder.

All commands

command	does
geistr catalog [--installed | --available] [--json]	the models, their state, fit and speed
geistr pull <id>	download, resume, verify
geistr pull	update the installed models after a geistr update
geistr run <model> [prompt…]	one answer to stdout; the prompt from stdin if none
geistr chat [<model>] [--new]	interactive; without a model the last one
geistr serve <model> [--http[=ADDR:PORT]] [--socket=PATH] [--chats N] [--max-request=MiB]	the model as a service
geistr chat --socket[=PATH]	chat with that service
geistr config [key [value]]	settings
geistr bench [model…]	tokens/s on CPU and GPU (default: every installed model)
geistr bench --compare [A [B]]	two engine versions' speeds side by side
geistr --version	geistr, catalog revision, engine commit

<model> is a catalog id or a path to a .gguf file. Options: --cpu / --gpu, --models DIR, --catalog FILE. Exit codes: 0 ok, 1 error, 2 usage, 130 cancelled.

Every complete answer records its speed in speed.tsv in the data folder; geistr catalog shows the median of the last ten.

Update

brew upgrade geistr, or run the install command again. The model list ships inside geistr, so new models come with a new geistr. Afterwards geistr catalog marks installed models whose file changed with ⟳ and a file of the right size with another checksum (damaged?) with ✗, and geistr pull fetches them (the old file stays until the new one is verified).

Python and C

geistr is built on geist-runtime, an embeddable model runner on geistlib with one C API (include/geistr.h) and a Python package (pip install geistr; every release also has its wheels at https://www.geisten.net/download/geistr/vX.Y.Z/):

import geistr

with geistr.chat("gemma4-e2b", system="Answer briefly.") as chat:
    for piece in chat.send("What is the capital of France?"):   # streamed
        print(piece, end="", flush=True)
    print(chat.ask("And of Italy?"))           # only the new message is processed

Using geistr: docs/GUIDE.md. Building, the library, tests and CI: docs/DEVELOPMENT.md. API reference and design: docs/API.md.