otaku 0.1.0

A terminal chat client for local LLMs — Ollama, LM Studio, MLX, and any OpenAI-compatible server.

otaku

A terminal chat client for Ollama, LM Studio, and any other OpenAI-compatible server. Single command, encrypted local history, full-screen model picker with load/unload, fuzzy-searchable conversation history.

Features

• One client for many backends — Ollama, LM Studio, omlx, and any server that speaks OpenAI's /v1/chat/completions and /v1/models.
• Interactive or one-shot — chat in a full REPL, or use it as a Unix filter: otaku llama3 "prompt" and cat file | otaku llama3 "explain" print a plain answer and exit, so otaku drops into any pipeline.
• Native model management — bare otaku opens a TUI picker that shows every model from every provider, marks the loaded ones, displays disk size, and lets you load/unload from the same screen. otaku stop unloads from the command line.
• Encrypted history — every message is AES-256-GCM-encrypted client-side before being written to a local SQLite database. The data key is wrapped by a key from your OS keychain (or a provider you choose) and never leaves your machine.
• Searchable history picker — /history (Ctrl+T) opens a two-stage picker (conversation → turn) over all your conversations, with a full-content filter (matches text anywhere in a chat, not just the summary or first prompt), summary preview, and "resume from any turn" semantics — prompting you to fork instead of overwriting when you resume before the last turn.
• LLM-generated conversation summaries — short, language-matching, keyword-rich summaries are computed by a background worker while you're idle at the prompt (never on exit — quitting is always instant, and a goodbye never reloads a cold model). They make /history searchable. Configurable via [defaults].create_summaries / summary_idle_seconds. Name a chat yourself with /title <text>; the title shows alongside (or instead of) the summary in the picker.
• Get answers out — /copy puts the last reply on your clipboard (native tool, falling back to the OSC 52 escape so it works over SSH); /save <file> writes the conversation to a Markdown file.
• Persistent defaults — a [defaults] config section plus per-model overrides (system prompt, think effort, parameters) so settings survive across sessions; /remember saves the current settings as the model's defaults.
• In-chat model switching — /model swaps the model mid-conversation (keeping context) — handy for comparing two local models on the same prompt.
• Slash commands — /clear, /new, /model, /regenerate (Ctrl+R), /undo (Ctrl+U), /history (Ctrl+T), /fork, /info, /print, /copy, /save, /title, /remember, /bye (Ctrl+D), /help (/?), /set system|think|verbose|parameter.
• Rendered markdown — replies stream with headers, lists, blockquotes, horizontal rules, and fenced code blocks, plus inline bold/italic/code. Code blocks are syntax-highlighted (via Pygments) with the language shown.
• Stats on demand — /set verbose on appends [ N tokens · R tok/s · Ts ] after each reply (off by default).

Requirements

• One or more model servers reachable over HTTP:
  • Ollama (ollama serve listening on :11434)
  • LM Studio (server enabled, listening on :1234)
  • …or anything else that speaks OpenAI v1 chat completions

History is stored in a local SQLite file (~/.otaku/history.db); there's no database to install or run.

otaku itself runs on Python ≥ 3.11. The recommended installer below fetches that for you if it's not on your system, so no separate Python setup is required.

Runs on macOS, Linux, and Windows. On Windows the one feature that needs a POSIX terminal — cancel-and-regenerate (Ctrl+R) while a reply is streaming — is disabled (everything else, including Ctrl+C to cancel, works); use WSL for full parity.

Installation

Two one-liners — pick your package manager:

# uv — pulls Python 3.11+ automatically, isolated in its own venv
uv tool install git+https://github.com/enclavum/otaku

# Homebrew — via the enclavum tap
brew install enclavum/tap/otaku

Either one puts otaku on your PATH in its own isolated environment, without touching your system Python. You just need a model server (Ollama, LM Studio, …) running — see Requirements. Update later with uv tool upgrade otaku / brew upgrade otaku; remove with uv tool uninstall otaku / brew uninstall otaku.

uv details — PATH setup and installing from a local clone

# Install uv if you don't have it
brew install uv
# or:  curl -LsSf https://astral.sh/uv/install.sh | sh

# Put uv's tool-bin directory on your PATH (one-time; restart your shell after)
uv tool update-shell

# Install from a local clone instead of straight from GitHub
git clone https://github.com/enclavum/otaku.git otaku
uv tool install ./otaku

First run

No database setup needed. The first time otaku runs it will:

• write a default config to ~/.otaku/config.toml
• generate an AES-256-GCM data key, wrap it with a key from your OS keychain (the default), and store the wrapped form in ~/.otaku/keys.json (mode 0600)
• create the encrypted SQLite history database at ~/.otaku/history.db

To use a passphrase, an external command, or a plain key file instead of the keychain, see Storage and encryption.

Quick start

otaku                       # pick a model and start chatting (interactive picker)
otaku llama3                # bare name — resolved across configured providers
otaku ollama/llama3         # explicit provider/model
otaku llama3 "2 + 2?"       # one-shot: print the answer and exit
cat err.log | otaku llama3 "explain"   # pipe a file in; combine with the prompt
otaku stop llama3           # unload a loaded model
otaku stop --all            # unload every loaded model in every provider

Inside the chat:

>>> Send a message (/? for help)

Type a message and press Enter. Type /? to see slash commands. Press Ctrl+T to switch conversations, Ctrl+R to regenerate the last reply, Ctrl+U to undo the last turn, Ctrl+D to exit.

Multiline input. To send a message that spans several lines (a stack trace, a code block, a multi-paragraph prompt), open it with """ and close it with """ — the same convention as Ollama's REPL:

>>> """
... def greet(name):
...     return f"hello {name}"
... """

The prompt switches to ... while the block is open; everything between the delimiters is sent as one message. A one-liner works too: """text""". Text wrapped in """ is always sent to the model verbatim, even if it looks like a slash command.

Usage

otaku (no arguments)

Opens the model picker; on confirm, immediately starts a chat with the chosen model. The picker remembers your last selection between invocations. Cancel in the picker exits without launching the REPL.

otaku <model>

Starts an interactive chat against a model — just pass it as the first argument (there's no run subcommand). The argument can be:

• Bare name (e.g. llama3, qwen3-coder-30b-a3b-instruct-mlx). otaku queries every configured provider's list_models and uses the unique match. If the same name shows up in two providers, otaku errors with a hint to disambiguate.
• PROVIDER/MODEL (e.g. ollama/llama3, ollama/hf.co/bartowski/SomeModel:Q8_0). The first segment must be a configured provider; the rest is the model name and may itself contain slashes.

otaku llama3                                       # bare; auto-resolved
otaku ollama/llama3:latest                         # explicit
otaku lmstudio/qwen3-coder-30b-a3b-instruct-mlx
otaku ollama/hf.co/bartowski/SomeModel:Q8_0        # nested slashes

(A model name that collides with a subcommand — list or stop — is treated as that command; use PROVIDER/MODEL to disambiguate.)

One-shot and pipes

Give a prompt as the second argument, or pipe content on stdin, and otaku runs a single completion, prints the plain reply to stdout, and exits — no REPL. This makes it a Unix filter:

otaku llama3 "write a haiku about pipes"          # prompt argument
git diff | otaku llama3 "write a commit message"  # piped content is the prompt
cat error.log | otaku llama3 "explain this error" # both: prompt + piped stdin
otaku llama3 "summarize" < notes.md               # stdin redirect

When both a prompt argument and piped stdin are present, they're combined instruction-first — the prompt, then a blank line, then the piped content (the same convention as mods):

write a commit message

<contents of the diff>

One-shot output is deliberately plain — no spinner, no markdown styling, no stats line — so it composes cleanly in a pipeline (… | otaku m "…" | pbcopy). The exchange is still saved to your encrypted history (use -nr to skip that).

otaku stop <model> / otaku stop --all

Unload a model so its weights leave RAM. Argument resolution mirrors run, but bare names match against the loaded set of each provider — so otaku stop llama3 unambiguously hits whichever provider currently has it loaded. otaku stop --all walks every provider and unloads everything that's loaded.

otaku stop llama3                # bare; resolved against loaded models
otaku stop ollama/llama3         # explicit
otaku stop --all                 # everything, everywhere

otaku list

list shows every model exposed by every configured provider — name, size, the loaded context window (CONTEXT, populated only for models in memory), and a ✓ in the LOADED column for those currently loaded. Add --running (-r) to filter to just the loaded models and drop the LOADED column — the equivalent of ollama ps, but spanning every provider you've configured (Ollama, LM Studio, etc.) in one table. Size and context columns are right-aligned so they read cleanly across rows.

$ otaku list
PROVIDER  MODEL                                 SIZE  CONTEXT  LOADED
ollama    llama3:latest                       4.7 GB       8K    ✓
ollama    mistral:latest                      4.1 GB        —
lmstudio  qwen3-coder-30b-a3b-instruct-mlx   17.5 GB     128K    ✓

$ otaku list --running
PROVIDER  MODEL                                SIZE  CONTEXT
ollama    llama3:latest                       4.7 GB      8K
lmstudio  qwen3-coder-30b-a3b-instruct-mlx   17.5 GB    128K

otaku -nr / otaku --no-record

When set, the session opens the database in read-only mode: nothing is written. Existing history stays browsable via /history, but the current session's turns and summary are not persisted, and deletions from the picker are blocked. Accepted both globally (before the model) and after it:

otaku -nr                          # bare picker, no-record session
otaku -nr ollama/llama3            # global flag, before the model
otaku ollama/llama3 -nr            # after the model

The REPL banner shows (not recorded — nothing saved) so it's obvious when you're in this mode.

Defaults and per-model settings

/set system, /set think, and parameters normally reset when you exit. To make them stick, otaku reads declarative defaults at launch (no surprising "sticky" auto-save):

• Global — a [defaults] section in ~/.otaku/config.toml:

  [defaults]
  system = "Be concise."
  think = "none"           # none | low | medium | high | max | default
  no_record = false        # open every session in --no-record mode
  create_summaries = true  # generate conversation summaries in the background
  summary_idle_seconds = 5 # summarize after this many seconds idle at the prompt
  [defaults.parameters]
  temperature = 0.7
  

• Per-model — overrides keyed by the bare model name, layered over the globals. Write them from a session with /remember (it saves the current system prompt, think effort, and parameters for the current model), or edit ~/.otaku/model_defaults.json by hand:

  {
    "deepseek-r1": { "think": "high", "parameters": { "temperature": 0.6 } }
  }
  

So a deepseek-r1 user runs /set think high once, /remember, and every future otaku deepseek-r1 starts with thinking on — while llama3 is unaffected.

Precedence, low to high: built-in defaults → [defaults] → per-model → -nr/--no-record flag → in-session /set.

Model picker

Used by the bare otaku flow. Loaded models render in bold black; not-loaded in muted grey. Models are grouped by provider, with a horizontal rule between each group. The header shows Models (N) on the left and RAM: used / total GB (N%) on the right.

Keybindings inside the picker:

key	action
↑/↓	move cursor
/	enter filter mode (Esc to clear)
l	load (with confirmation + spinner dialog)
u	unload (with confirmation + spinner)
Enter	save selection to ~/.otaku/last_model and exit; also loads if not loaded
Esc	quit without saving

REPL slash commands

/clear                       Clear context, stay in the same conversation
/new                         Clear context and start a new conversation
/model [PROVIDER/MODEL]      Switch model in-place (opens the picker with no arg)
/undo                        Discard the last prompt and response (Ctrl+U)
/regenerate                  Re-run the last prompt (Ctrl+R)
/history                     Browse saved conversations and resume any turn (Ctrl+T)
/fork                        Snapshot the current conversation as a new branch
/info                        Show details about the current model + session
/print                       Dump the full message history (what the model sees)
/copy [all]                  Copy the last reply (or the whole chat) to the clipboard
/save <file>                 Save the conversation to a Markdown file
/title <text>                Name this conversation (shown in /history)
/remember                    Save current system/think/params as this model's defaults
/set verbose on|off          Show the stats line after each reply (off by default)
/set system <text>           Set the system prompt
/set think <level>           Set thinking effort: on|off|none|low|medium|
                             high|max|default. Off by default. `off`
                             actively disables thinking; `default` removes
                             the request so the model uses its own default.
                             How it's sent is provider-specific (Ollama:
                             reasoning_effort; omlx: enable_thinking).
/set parameter <name> [val]  Set or clear an inference parameter
                             (temperature, top_p, max_tokens, presence_penalty,
                             frequency_penalty, seed, stop)
/bye                         Exit (Ctrl+D)
/?, /help                    Show this help

Keys at the prompt:

Key	Effect
"""	Begin a multiline message; close it with """
Tab	Complete slash commands
Up/Down	Walk this session's prompt history
Ctrl+R	/regenerate last response (during stream: cancel + regenerate)
Ctrl+U	/undo the last turn
Ctrl+T	Open /history picker
Ctrl+D	/bye (also exits on empty line)
Ctrl+C	Clear the current line; cancel an in-flight reply

/history picker

Two-stage picker:

1. Conversation list — each row shows title / summary (or just one of them; (untitled) if a chat has neither). Press / to start filtering; the preview pane on the right shows the model, last-updated timestamp, title, summary, and first prompt. Set a title with /title <text>.
2. Turn list — pick the turn to resume from. The conversation is loaded up to (and including) that turn.

If you resume from a turn that isn't the last one, continuing the chat would overwrite the later turns on the next save. otaku catches this and prompts:

Resuming at turn 5 of 12 would discard 7 later message(s) on next save. Fork into a new conversation? [Y/n]

Press Enter (or y) to fork into a fresh conversation containing just the turns up to your pick — the original is left untouched. n resumes destructively (later turns are lost on the next save). The prompt is skipped in no-record (-nr) sessions, where nothing is written anyway.

Press Del on a row in the conversation list to delete it (after a confirmation modal).

Configuration

~/.otaku/config.toml:

[database]
url = "sqlite:///~/.otaku/history.db"

[encryption]
provider = "keychain"   # keychain | passphrase | command | disk

[providers.ollama]
url = "http://localhost:11434/v1"
api_key = ""
supports_thinking = true
keep_alive = "24h"

[providers.omlx]
url = "http://localhost:8000/v1"
api_key = ""
supports_thinking = false
smooth = true

[providers.lmstudio]
url = "http://localhost:1234/v1"
api_key = ""
supports_thinking = false

The [encryption] block selects where your encryption key lives; keychain (the OS keychain) is the default and needs no other setup. See Storage and encryption for the other providers.

Add new providers by adding more [providers.<name>] sections. Each one needs a url (the OpenAI-compat base, including /v1); api_key, supports_thinking, and keep_alive are optional.

Set OTAKU_DATABASE_URL in the environment to override [database].url for a single run (handy for pointing at a throwaway DB without rewriting the config):

OTAKU_DATABASE_URL="sqlite:////tmp/otaku-test.db" otaku

keep_alive (default "24h") is forwarded to ollama's /api/generate on the explicit load action from the model picker — it sets how long the model stays loaded in memory. Standard ollama duration syntax: "5m", "24h", 0 (unload immediately after request), -1 (never auto-unload). Other providers ignore the field.

smooth (default true, omlx only) de-jitters omlx's output. omlx batches tokens before flushing (a throughput optimization in its serving engine), so its stream arrives in bursts and looks choppy even though it's not slow. With smooth on, otaku buffers the incoming bursts and reprints them at a steady pace, so you get smooth token-by-token typing. It adds a small (~150 ms) display lag; set smooth = false to print bursts as they arrive. Other providers ignore the field.

Provider auto-detection: otaku probes each provider's URL on first use to decide whether it's Ollama (/api/ps), LM Studio (/api/v1/models with {key, size_bytes, loaded_instances}), omlx (/v1/models/status with per-model loaded flags), or a generic OpenAI-compat server. The detected backend determines whether the picker's load/unload actions (and otaku stop) are available. Probes use a short (0.5s) timeout and every command that spans providers queries them concurrently, so one configured-but-down provider doesn't slow the others down. When nothing is reachable, otaku lists each provider, whether it answered, and points at ~/.otaku/config.toml to fix.

Storage and encryption

• History lives in a local SQLite database at ~/.otaku/history.db (WAL mode). No server, nothing to install, and it never leaves your machine.

• All conversation messages, summaries, and titles are encrypted client-side with AES-256-GCM before being written to disk. The database never sees plaintext.

• Keys use an envelope scheme: a random 32-byte data-encryption key (DEK) encrypts the blobs; the DEK is itself wrapped by a key-encryption key (KEK), and only the wrapped form is stored, in ~/.otaku/keys.json (mode 0600). The DEK never touches disk in the clear.

• The KEK comes from the provider you set in [encryption]:

  provider	where the KEK lives
  keychain	your OS keychain (macOS security, Linux secret-tool). Default.
  command	fetched by [encryption].retrieve_command — 1Password op, pass, gpg, … (must print base64 of 32 bytes to stdout).
  passphrase	derived from a passphrase via scrypt; prompted each launch, nothing stored.
  disk	plaintext ~/.otaku/kek.key (0600); the automatic fallback when no keychain tool is found.

  On first run otaku generates the KEK (or, for command, prints one for you to store). Switching providers is not yet automated — it currently means starting from a fresh ~/.otaku.

• Back up ~/.otaku/keys.json and your KEK (the keychain item, or whatever retrieve_command reads). Either alone is useless; losing the KEK makes your history permanently unreadable.

• Schema bootstrap is idempotent (CREATE TABLE IF NOT EXISTS). Two tables: conversations and messages.

• ~/.otaku/last_model records the model selected from the picker (one line, provider/model).

Provider support matrix

feature	Ollama	LM Studio	omlx	other OpenAI-compat
chat	✓	✓	✓	✓
model picker	✓	✓	✓	✓ (no load/unload)
disk size shown	✓	✓	✓	—
load / unload	✓	✓	✓	—
reasoning_effort	✓ *	(model)	(model)	(model)

*ollama needs a supports_thinking = true flag in the provider config to forward the reasoning_effort parameter.

Development

See CONTRIBUTING.md for development setup, the lint, type-check, and test workflow (pytest — a full unit + CLI suite in tests/), and how to smoke-test against a real backend.

License

MIT.