yapbox

A persistent-presence voice agent. The mind runs continuously on your machine; the voice is just a channel that opens when you speak and closes when you are done.

Most voice-agent frameworks orchestrate a single live conversation. yapbox is a different thing: a long-lived brain (durable cognitive state, a write-ahead journal, a job ledger, cross-session memory) that happens to talk. Say the wake word and an ElevenLabs session opens; say you are done and it closes. Jobs keep running while the channel is shut, state keeps accruing, and the next wake opens with a pre-computed greeting that already knows what happened in the gap. Persistent mind, ephemeral socket.

"Hey Silas" is the shipped default persona a fresh clone meets. Two more ship in personas/: Jarvis (a dry-butler register) and Penny. A persona is a voice plus a soul, a markdown character brief with worked examples, and the wake keyword selects which one opens. See personas/README.md for the soul contract and docs/persona-interview.md to build your own.

Why it is different

yapbox is built for one job: an always-on voice presence that can safely drive your machine, without burning money or going silent unnoticed. The things below are the product; the voice is the mouth and ears.

• Persistent mind. A brain daemon holds cognitive state across sessions (journal plus atomic snapshot plus boot reconciliation), not a per-call stitch that dies with the session.
• Durable jobs. A separate orchestrator runs work on a SQLite ledger in a crash-isolated subprocess pool. A job survives a brain restart and gets narrated when it lands.
• Cost-safe by construction. Nested fail-closed budget breakers and a spend ledger. The voice session is metered, so it opens on a wake word and closes on idle rather than staying open.
• Bring your own backend. The work harness (Claude Code, Hermes, Codex), the memory layer, and the toolchain (MCP servers, CLIs) are all pluggable. Memory ships as flat-file JSONL, the working zero-config default; a Honcho backend is in the tree behind the same interface but its live transport is not wired yet, so treat it as experimental. Machine control (shell, browser, computer-use) is double-opt-in and gated.
• Always-on, supervised. A launchd supervisor restarts crashes with exponential backoff; a separate deadman alerts out-of-band if the supervisor itself goes quiet.
• Mock-safe. Around 1000 tests run with no live keys. Nothing spends money or opens a port until you flip an explicit env var.

Architecture

Three supervised OS processes, one uv-pinned Python 3.12 venv. A launchd supervisor owns them (crash-only, exponential-backoff restarts); a separate deadman cron alerts out-of-band if the supervisor itself goes quiet.

supervisor.py (launchd KeepAlive)        deadman (separate label, 30s)
   |
   +- P1 brain         one asyncio loop owns everything voice-shaped:
   |                   mic duplex + echo cancellation, sherpa-onnx wake word,
   |                   the EL session, CognitiveState (journal + atomic
   |                   snapshot), the custom-LLM SSE server, flash TTS for
   |                   unprompted lines, fail-closed cost breakers
   |
   +- P2 orchestrator  localhost job runner. SQLite job table, crash-isolated
   |                   subprocess pool (claude -p, codex, shell, hermes, ...),
   |                   202 + poll contract, push to the brain on job-done.
   |                   Survives a brain restart; jobs finish anyway and get
   |                   narrated on reconnect.
   |
   +- P3 surface       read-only operator glass cockpit, 127.0.0.1 only,
                       bearer-gated. Live status, telemetry feed, open jobs.

The audio half of P1 sits behind an AudioBackend protocol (start/stop/output/interrupt plus a frame callback). A native capture shell can replace the Python audio half over a unix socket (line-delimited JSON, contract in src/yapbox/ear/SEAM.md) without touching the brain. The default Python half (sounddevice plus APM echo cancellation) passed a 50-minute soak, so the shell stays optional; the seam stays tested.

Modes

Mode B is the supported path, and it is what you run out of the box: ElevenLabs hosts the conversation LLM and reaches your brain through six client tools (dispatch, check, recall, narrate, remember, cancel). No public URL, no tunnel, works with any voice.

Mode A is the advanced, lowest-latency option: ElevenLabs drives the conversation against your brain's own custom-LLM endpoint through a cloudflared tunnel, giving full prompt control behind one public URL with a mandatory bearer. The mode-A endpoint code (llm_server.py) is in the tree, but the tunnel process that put it on a public URL was retired with the native-EL switch and is being rebuilt, so mode A is not a live opt-in today. mode = "auto" prefers A when a healthy tunnel is present and otherwise selects B at the session boundary; with no tunnel running it always selects B. (Instant-clone voices are mode B only.)

Quickstart

uv sync
bin/yapbox up        # supervisor + brain + orchestrator + surface, waits on heartbeats
bin/yapbox status    # heartbeat ages for each process
bin/yapbox test      # the full mic-less suite (~1000 tests, fixtures, no network)
bin/yapbox demo      # prints the five-beat walkthrough script (a runnable demo is in progress)
bin/yapbox down      # clean teardown, no leftover PIDs or ports

The default posture is mock-safe: null audio backend until you configure a mic device UID, EL sessions refuse to open, flash TTS dry-runs, the operator surface binds loopback only, the deadman runs dry. Nothing spends money or exposes a port until you flip an explicit env var: YAPBOX_AUDIO=sounddevice, YAPBOX_EL_LIVE=1, YAPBOX_TTS_LIVE=1 (flash TTS for unprompted lines; otherwise it dry-runs and charges nothing), YAPBOX_OWL_LIVE=1. Each flip is deliberate; none is default. (The legacy ALBUS_* env names are still read as a back-compat fallback.)

To go live you need one thing: an ElevenLabs API key and a voice. Drop them into a persona config under ~/.config/yapbox/<persona>/config.toml, provision the agent (scripts/provision_agent.py --persona <name>), and bin/yapbox up.

Docs

• docs/QUICKSTART.md - from clone to first wake word, step by step
• docs/ARCHITECTURE.md - the process model, cognitive loop, and durability doctrine
• personas/README.md - the soul-file contract every persona satisfies
• docs/persona-interview.md - a paste-ready interview that emits a contract-valid soul
• CONTRIBUTING.md - dev setup, test posture, and how to add a persona or a backend
• src/yapbox/ear/SEAM.md - the audio-seam wire contract for a native capture shell