hermes-talk

Realtime duplex voice for your own agent — talk to it, it runs real work in the background, it reports back out loud.

It rides your ChatGPT or SuperGrok subscription. No API key required.

Talk runs in the terminal (hermes talk), Discord voice (/talk join), and the Hermes dashboard Talk tab. It calls the host's tools, delegates background work while you keep talking, reports results, and handles current approval requests. Provider support differs by surface; see the tables below. Built as a plug-in for Hermes Agent.

New in 0.18.0: GPT-Live with separate subscription and API billing, plus Codex workers attached to a selected Hermes task. Subscription is the default, with no automatic paid fallback. These features require the compatible task-worker host. Setup, controls and verification: GPT-Live and task workers.

hermes plugins install TheSmokeDev/hermes-talk --enable && pip install "hermes-talk[audio]" && hermes talk

🔊 Watch the 2:27 cut with sound — delegate, keep talking, hear the result land.

Real Realtime session, 8× speed in the GIF, recorded at v0.3.0. It does not demonstrate the new GPT-Live or Codex-worker integration.

Is it working?

hermes plugins list        # → hermes-talk · enabled · current version
hermes talk --help         # → registration proof: the command only exists if the plugin loaded
hermes talk setup          # → guided, confirmation-gated setup plus doctor verification
hermes talk doctor         # → read-only diagnostics: auth lane, provider, model/voice, audio, host lanes
hermes talk doctor --json  # → the same versioned receipt for scripts and issue reports
hermes talk check          # → prove it: doctor + one live provider turn + one bounded Hermes run
# then, in any session: say "status report" — talk_status answers with
# version, auth lane, agent lane, and audio state.

Doctor is read-only by design: it names which lane came up and what is missing, and never writes, probes, or refreshes a token (the one exception is --probe, Grok-only, which makes two live calls to api.x.ai and says so).

Check the existing provider lane — hermes talk check

check exercises the existing TALK_PROVIDER adapter and a bounded Hermes run. It does not certify GPT-Live, Codex-worker handoffs, or microphone behavior. Use the Live acceptance checklist for those.

A green doctor can still hide a dead mint, a refused socket, or a delegation lane that never starts. check is the other half: it runs the doctor checks, then opens a real session on your configured provider through the same adapter and credential path the voice uses (connect → SessionReady → one text turn → ResponseFinished), then hands one bounded task to a real Hermes agent through the same delegation path the voice uses, and only passes if the agent's output contains HERMES_TALK_CHECK_OK.

hermes talk check              # three steps, one verdict, exit 0 only if every step passed
hermes talk check --json       # per-step pass/fail/skip with durations, for issues and scripts
hermes talk check --no-run     # provider session only; skip the Hermes run
hermes talk check --timeout 60 # budget for the run step (default 180s; provider turn 60s)
hermes talk check --provider grok  # check a live lane other than TALK_PROVIDER (this process only)

What it guarantees: every live step has a hard wall-clock bound (a run that outlives its budget is stopped, not abandoned); the report carries no tokens and no paths; a failing doctor check skips the live steps rather than spending a session on a known-broken config; and a mock can never go green — --provider accepts only live lanes, the report's provider is validated against the same fail-closed list, and the live steps refuse under a test harness. It is not read-only: expect one short provider turn and one short agent run (recorded in the run history like any other), credential resolution exactly as a session does it, and nothing else changed. Paste hermes talk check --json into a bug report — it says which half is broken.

Filing an issue? hermes talk diagnostics --bundle writes one redacted, owner-only file — versions, the names of the variables you have set, device and host facts, and every doctor outcome; no values, logs, prompts, transcripts, audio, or paths — through a default-deny allowlist, so it is safe to paste into a public issue and it is what the bug template asks for.

Upgrade with hermes plugins update hermes-talk — not a second install (it refuses on an existing plugin) — then restart the gateway: a running process keeps executing the old code until you do. Full runbook, wire canary included: docs/OPERATING.md.

Surfaces

Providers

TALK_VOICE_MODE=live selects GPT-Live separately from this table. TALK_LIVE_AUTH=subscription is the default; api must be selected explicitly. Subscription failure never falls back to a paid API key. Model and voice settings are separate for each billing option; see GPT-Live configuration.

The provider knob is fail-closed and never inferred from which keys exist. Full per-lane detail: Provider details.

What this actually is

Not dictation. Not read-my-reply-aloud. A conversation you can hand work to while it's still going:

You: audit the auth module for error-handling gaps and report back Hermes: starting that now — run one. You: while that runs — what did we decide about the retry policy? Hermes: (searches your past sessions) three attempts with exponential backoff, decided on the 14th… You: how's that audit going? Hermes: run one's still working, about two minutes in. …later, unprompted: Hermes: that audit finished — three gaps, starting with the token refresh swallowing exceptions…

Four properties make that possible, and each one is the part other voice integrations don't have:

1. Duplex. One bidirectional audio session — turn-taking, interruption, and tool calls happen inside the speech layer, not around it. Cut it off mid-sentence and it stops, because it never stopped listening.
2. Its tools are Hermes's tools. Realtime function calls relay straight into the agent's real tool surface. Ask it something it can't know and you hear it go look, then answer from what it found.
3. Work outlives the sentence. Delegation spawns a real background Hermes agent. You keep talking. The result is spoken when it lands — you don't poll, you don't wait, you don't go check a terminal.
4. It starts already knowing you. The session prompt is assembled from what Hermes itself knows — your SOUL.md, and whatever your configured memory provider contributes. Install one (e.g. hermes-homie-memory) and the first thing you say lands on an agent that already has context. No tool call, no "let me look that up", no warm-up turn.

Hermes's built-in voice mode is good and this doesn't replace it — turn-based STT → inference → TTS is the right shape for plenty of work. This is the other shape.

Install

Needs Python ≥ 3.11 and a Hermes host ≥ v0.17. redirect_agent's clean-abort path wants 0.20+ and degrades honestly below it. The plugin remains backward-compatible with older hosts, but session-owned subagent completion announcements require a Hermes release exposing PluginContext.active_parent_session_id (upstream PR #79716); without that property, announcements are suppressed rather than guessed. Details are in docs/OPERATING.md.

hermes plugins install TheSmokeDev/hermes-talk --enable
pip install "hermes-talk[audio]"   # mic + speaker support (sounddevice); skip if dashboard-only
hermes talk

The legacy voice lanes retain their older-host compatibility. Shared task attachment, GPT-Live handoffs and Codex workers additionally require the host capabilities listed in GPT-Live prerequisites. Updating the plugin alone does not add those host capabilities. Offline tests are in tests/; CI covers Ubuntu and Windows, Python 3.11 to 3.13.

Quickstart — your first call on each surface

These examples use the existing provider lanes. For GPT-Live, an explicit task and the updated host are required; follow the Live guide.

Terminal (simplest — start here):

hermes plugins install TheSmokeDev/hermes-talk --enable
pip install "hermes-talk[audio]"
hermes talk        # you are live; speak. Ctrl+C hangs up.

Discord (the call happens inside a voice channel, not in chat):

1. Hermes's Discord adapter is connected and its bot is in your server (that's the host's own setup, not this plugin's).
2. Put Hermes in a voice channel first: /voice join. Talk borrows that connection — it never opens a second one.
3. Then /talk join starts the Talk session on that channel. /talk leave (or leaving the channel) ends it. /voice leave disconnects Hermes entirely.
4. Want mutating tools (delegate_task, steer_agent, …) in a shared channel? Set TALK_DISCORD_OPERATOR_USER_IDS=<your Discord user id> and restart the gateway. Without it everyone can talk but nobody can mutate — that fail-closed default is intentional.

Dashboard (browser, no mic drivers needed): with the gateway running, open the Hermes dashboard and use the Talk tab — start the session from there. Audio is browser-native.

In every case, hermes talk doctor (or saying "status report" on the call) tells you which lane came up and what is missing. If a surface shows nothing, run doctor first — it names the gap.

Auth — no API key needed if you have ChatGPT

This section describes OpenAI Realtime. GPT-Live uses the independent TALK_LIVE_AUTH selection described here.

Signed into the Codex CLI (codex login)? Talk runs on your own ChatGPT subscription's Realtime entitlement — no key, no per-minute API bill. Bring a key instead if you'd rather.

Resolved fail-closed in this order:

1. TALK_OPENAI_API_KEY — a Talk-scoped API key (set-but-empty refuses, never falls through)
2. OPENAI_API_KEY — the shared environment key
3. Codex OAuth — no key at all: if you're signed into the Codex CLI (codex login), Talk rides your own ChatGPT subscription's Realtime entitlement. If Hermes itself is logged in (hermes auth login openai-codex) that login is borrowed first, with the host handling refresh. Talk reads ~/.codex/auth.json but never refreshes or rewrites it; an expired token asks you to codex login again.

That historical order remains unchanged when TALK_PREFER_CODEX_OAUTH is absent or explicitly false. Set TALK_PREFER_CODEX_OAUTH=true to require the subscription lane even when API keys exist. The preference is fail-closed: a missing/unusable Codex login refuses instead of spending a metered key, and a blank or invalid preference refuses until corrected. hermes talk doctor names the winning lane and distinguishes valid OAuth from an expired credential that needs a fresh codex login; it never prints the key or token. When setup offers the API-key lane under an enabled OAuth preference, it reuses an existing metered key when present and separately confirms the required TALK_PREFER_CODEX_OAUTH=false policy transition.

For these Realtime lanes, Talk mints an ephemeral client secret for the audio connection. GPT-Live uses its own server-owned negotiation and sideband; provider keys, OAuth tokens and account IDs are never sent to the dashboard browser.

Provider details — OpenAI (default), Grok, or Gemini

TALK_PROVIDER picks the realtime voice transport: openai (default, everything above), grok (xAI Grok Voice), or gemini (Gemini Live). The knob is fail-closed and never inferred from which keys exist — an operator holding several gets the provider they named or an error, not a silent switch.

The Grok lane runs on an X subscription — no key: hermes auth add xai-oauth once, then TALK_PROVIDER=grok. Verified live on X Premium (the $8 tier); a tier without realtime access is told so in one line rather than a traceback. Talk consumes the host's xai-oauth login the way the OpenAI lane consumes the Codex CLI's (the host refreshes and stores; Talk never writes an auth store). Bring an xAI key instead if you'd rather (TALK_XAI_API_KEY, falling back to XAI_API_KEY; set-but-blank refuses). Keys win over the login unless TALK_PREFER_XAI_OAUTH=true, which requires the subscription and refuses metered fallback, fail-closed like its Codex twin. A rejected or tier-denied token gets a one-line remediation at connect, never a traceback; hermes talk doctor --probe makes two live calls to api.x.ai to prove the resolved bearer reaches realtime before you sit down to talk. The lane rides model grok-voice-latest (override: TALK_GROK_MODEL), and offers five voices — ara, rex, sal, eve, leo — via TALK_GROK_VOICE (fail-closed on unknown names). Same contract, same tools, same barge-in; terminal and Discord lanes both honor the knob. The dashboard tab stays OpenAI-only for now — xAI has no WebRTC offer endpoint, so that lane is a separate backend-relay piece. Doctor gains a provider check: selection, redacted key presence, model/voice validity.

The Gemini lane is the zero-cost option: free-tier Google AI Studio keys work. Set GEMINI_API_KEY (or Talk-scoped TALK_GEMINI_API_KEY; set-but-blank refuses), ride model gemini-3.1-flash-live-preview (override: TALK_GEMINI_MODEL), and pick a voice via TALK_GEMINI_VOICE — Puck, Charon, Kore, Fenrir, Aoede, fail-closed and case-sensitive, exactly as Google's wire expects them. Two lane-specific notes: the key rides the WebSocket URL query on this provider, so the URL is treated as a secret (assembled at connect, never logged, scrubbed from transport errors), and the Live protocol has no client cancel/truncate command, so barge-in bookkeeping degrades to local playback handling with a logged receipt — never a faked upstream call. The Discord lane refuses Gemini for now: its gated-response authorization flow has no Live wire equivalent, so connect fails closed rather than answering unvetted speakers.

Hermes core realtime contract

Hermes core now defines its own provider-neutral speech-to-speech contract (agent/realtime_voice_provider.py, API v2 — NousResearch/hermes-agent#101808) and drives any registered backend from hermes realtime. hermes-talk publishes all three of its lanes there, so the same OpenAI, Grok, and Gemini sessions this plugin already speaks are available to core's own orchestrator:

hermes realtime --list
# hermes-talk/openai   ready        OpenAI Realtime (hermes-talk)
# hermes-talk/grok     needs setup  xAI Grok Realtime (hermes-talk)
# hermes-talk/gemini   ready        Gemini Live (hermes-talk)

hermes realtime --provider hermes-talk/gemini

There is nothing to configure. The registration is feature-detected: on a Hermes that does not expose the contract — every released version today — hermes-talk loads exactly as it always has, with one debug line and no warning. The names are namespaced so they can never shadow core's bundled openai.

Capabilities are declared, never faked. Core asks each session what it can do and degrades explicitly rather than being told a comfortable lie:

¹ Grok does put conversation.item.truncate on the wire; if the server answers that it is unimplemented, that session degrades to cancel-only and says so once.

Gemini Live has no client-side cancel, no output truncate, and no conversation-item delete, so it advertises none of them and core drops playback locally on barge-in instead of being handed a truncation that never happened. What it has and the others do not is toolCallCancellation, which now reaches the host as a real cancelled-tool event — so results for a call the server retracted are never submitted.

Auth is unchanged, and readiness stays offline: hermes realtime --list reads each lane's own read-only diagnostic and never opens a socket, refreshes a token, or writes an auth store.

Custom voice — the cascade lane (ElevenLabs)

Native mode speaks with the provider's own voices, and those voices are provider-locked. Cascade mode splits the call: the realtime provider stays the brain (listening, thinking, tools, turn-taking) and hands its answer TEXT to a streaming ElevenLabs TTS, so the assistant speaks in any voice on your ElevenLabs account — including a clone of your own.

TALK_VOICE_MODE=cascade \
TALK_ELEVENLABS_VOICE_ID=<your-voice-id> \
hermes talk

The key comes from TALK_ELEVENLABS_API_KEY or ELEVENLABS_API_KEY (Talk-scoped wins; set-but-blank refuses), and the TTS model defaults to eleven_flash_v2_5 (override: TALK_ELEVENLABS_MODEL). To clone your own voice, create it in ElevenLabs VoiceLab first (VoiceLab → your voice → copy the ID); voice management stays in your ElevenLabs account, not the plugin.

The trade, stated plainly: native provider audio starts ~300–600ms after turn end, and the cascade adds roughly one extra half-second on the FIRST sentence (sentence chunking plus TTS first-audio, ~490ms measured) — later sentences pipeline under playback. You trade ~0.5s of first-word latency for your voice.

Cascade is OpenAI-only for now (it is the one provider whose text-output mode is wired and verified — picking grok or gemini fails closed and names the provider). Barge-in cuts the cloned voice off exactly like native: SpeechStarted aborts the in-flight TTS stream and drains playback in the same synchronous step, so a cancelled sentence never speaks. A TTS failure degrades that one answer to text-only with a single logged receipt; the call itself survives. TALK_VOICE_MODE is fail-closed and defaults to native, which is byte-identical to the pre-cascade behavior. Doctor gains a cascade check: mode, TTS provider, redacted key presence, voice-id status — no live probe.

The cascade speaks on every Talk surface:

Use

hermes talk       # terminal duplex voice session
hermes talk setup # detect → ask only missing decisions → confirm/write → verify
hermes talk doctor # strictly read-only configuration and host diagnostics
hermes talk check  # doctor + one live provider turn + one bounded Hermes run; exit 0 = proven
hermes talk diagnostics --bundle  # redacted support bundle for issue reports, written owner-only

Setup commits all individually confirmed settings to the active Hermes home's .env as one secure atomic transaction and updates the current process to match. On failure it rolls both surfaces back when possible, emits a value-free applied/rolled-back/failed receipt, attempts and verifies every secret-bearing temporary-file cleanup, and reruns doctor whenever a mutation may remain. Any surviving temp is a surviving mutation: setup returns failed and identifies the cleanup slot/error class without printing the path nonce or secret value. New secret files keep POSIX 0600 behavior and receive a protected owner-only DACL on Windows; an existing Windows destination DACL is preserved. A healthy configuration asks no questions and performs no writes. Doctor never delegates to setup.

or /talk inside an interactive Hermes session, which additionally reaches the agent-loop-only tools (memory, session_search, honcho_search, delegate_task). A spoken memory lookup tries the transcript first (session_search) and remembered profile facts second (honcho_search), and says which of the two it answered from — a recollection can be stale in a way a verbatim line cannot, and nothing is on screen to check it against.

Pause the microphone without hanging up. Say "stop listening" (or "mute the mic") and the model calls pause_voice_input: the call stays connected, playback keeps playing, background work keeps running and its results are still announced — only your speech stops reaching the provider. A paused microphone cannot hear the word "resume", so the way back is your own control, and the pause is offered only where that control is guaranteed to exist:

• hermes talk in a real terminal — Enter toggles (p and r are explicit; on Windows they are single keys, elsewhere type the word and Enter). The connected line says Enter to pause or resume the microphone when the key is live. With a piped or non-tty stdin (Git Bash's mintty reports no tty to Python; launcher wrappers) there is no key, so no pause is offered and a pause call is refused with a receipt that says why.
• /talk typed at the Hermes prompt — the prompt owns that terminal for the whole call, so the session never watches it for a key, and offers no pause either. Use hermes talk on its own when you want the control.
• Discord — /talk pause and /talk resume, typed. The model-side tool is offered on the legacy provider-owned lane; on the provider-host-tools lane the host supplies the tool list and the typed commands are the path.

Both directions get a spoken receipt, the receipt names the control for the room you are in, and Ctrl+C still hangs up.

In Discord, /talk join runs the call in the voice channel Hermes is already in — same conversation, same tools, same steering, in a room other people can hear. Talk now reports speaker transitions to the model using the member's immutable Discord user ID; display names are quoted as untrusted data, and an unknown SSRC stays unresolved and unauthorized. Configure immutable IDs with TALK_DISCORD_OPERATOR_USER_IDS=<id>[,<id>...]. Only those speakers may run delegate_task, steer_agent, redirect_agent, or stop_work; everyone may still converse and use read-only tools — including pause_voice_input, which can only narrow what the session does; /talk resume (text) brings listening back. Unset, blank, or any malformed list authorizes nobody. Talk binds permission to the exact Discord PCM, VAD input item, and opaque Realtime response metadata — never a display name, SSRC, model argument, or whichever person spoke most recently. Mixed, missing, or unresolved attribution fails closed with a spoken denial. Terminal microphone and dashboard sessions retain their existing behavior. Talk borrows the host's own voice connection rather than opening a second one. Details: docs/OPERATING.md.

What can you actually say? The full say-this → hear-this card, with what each spoken receipt commits to: docs/VOICE-COMMANDS.md.

Dashboard tab

The demo at the top of this README is this tab. Start the dashboard and Talk appears in the nav:

hermes plugins enable hermes-talk   # already done by `install --enable`
hermes dashboard                    # then open the Talk tab

Hit Start, allow the microphone, and talk. The page mints an ephemeral secret server-side, dials OpenAI directly over WebRTC, relays every function call back into the plugin's real tool surface, and shows the transcript plus a live list of background runs. Nothing to install — the bundle ships with the plugin and the host serves it.

Memory writeback currently covers terminal and Discord Talk sessions. Those rooms share the server-side Realtime relay, which durably captures completed turns. The dashboard's Realtime events stay in the browser, so matching durable capture requires a separate authenticated transcript endpoint; until that lane exists, the tab does not claim to write its conversation back to memory.

The tile at the top of the tab reads attached, api-server, or out of process — which of the three agent lanes below this session would actually use. It is not a guess; it is the lane the next tool call will take.

TALK_DASHBOARD_TOKEN — the tab's own gate

Dashboard routes already sit behind the dashboard's session auth, but this plugin's routes mint real credentials, so they carry a second check that never fails open:

• Unset (default): loopback only. A browser on the same machine works with no configuration. Anything else is refused with a message naming this variable — including a request whose peer address this process cannot read at all, which is treated as remote rather than trusted.
• Set: the token is required, on loopback too, compared with hmac.compare_digest. Paste it into the field the tab offers when it gets refused; it's held in sessionStorage, so it dies with the tab.

Set it whenever the dashboard is reachable from anywhere but this machine.

Reaching a real agent — the three lanes

Everything that needs an actual Hermes agent — a memory lookup, a delegated task — goes down the same chain, and every fall-through is said out loud:

1. Attached — the agent loop this session is running inside. Only /talk has one. Answers come back inline, in the same breath.
2. api-server — a real, fully-tooled Hermes agent reached over the api_server gateway platform. This is what makes the dashboard tab and a standalone hermes talk more than a fallback.
3. Out of process — no agent lane. Delegation still spawns a detached hermes -z one-shot; a memory lookup refuses, naming exactly what's missing.

Lanes 2 and 3 answer with a receipt rather than the answer, and speak the result when it lands. That is not a shortcut: an agent run takes seconds to minutes, and the tool call that starts it runs on the same thread carrying your microphone. Waiting there wouldn't be patience, it would be dead air.

Turning the api-server lane on

# in your gateway environment (HERMES_HOME/.env works)
API_SERVER_KEY=<a strong key you choose — 16+ characters>

Restart the gateway. The platform enables itself when a usable key exists — current hosts require the key (an unauthenticated api-server refuses to start), so API_SERVER_ENABLED alone does nothing. Talk finds it by itself — no Talk-side configuration is needed, because API_SERVER_KEY is the same variable the gateway reads. If you want Talk to use a different key or a non-default address, set TALK_API_SERVER_KEY / TALK_API_SERVER_URL.

Talk probes GET /v1/capabilities (which is authenticated, on purpose) at session start, so a wrong key is reported as "running but rejected my key" rather than as "not reachable" — those send you to two different places.

Background work

Say "go audit the site and tell me what's broken" and it starts a real agent, then keeps talking to you. When the work lands, Talk speaks the result unprompted. Ask "how's that going?" in the meantime and check_work answers.

Between the receipt and the landing, the session speaks bounded progress milestones: "accepted", "executing — Reading files", "waiting on an approval", and periodic "still working" heartbeats. The only detail that can name what a job is doing is a safe label from a fixed table — never the tool's arguments, paths, or output.

Known limitation: delivery is bound to the exact session that started the work. If you disconnect before it lands, reconnecting on the same Hermes session adopts and speaks what you were owed, exactly once — a different session, or one with no durable Hermes context, never receives it.

Delegation walks the three lanes and then one more, and every fall-through is said out loud — the plugin never silently does less than you asked:

1. Hermes's own agent loop — inside /talk, where there's a parent agent to delegate into.
2. A real agent over the api_server — preferred over a spawn: it reuses a warm, fully-tooled agent instead of paying a process start.
3. A detached hermes -z one-shot — needs nothing enabled, so this is the lane that always exists as long as hermes is on the PATH.
4. None available — a refusal naming all three missing lanes.

Two jobs, one checkout — admission control

Delegate two tasks that both touch the same repository and, left alone, they race: two agents editing one checkout, two deploys to one target. Since #101 the model can say what a task touches, and the run registry refuses the second job instead of letting it collide:

• delegate_task takes two optional arguments. resource_keys names up to eight stable things the task touches — an absolute repo path, a deployment target, a service name (whitespace-collapsed and case-folded, so two spellings of one path are one key). execution_mode is exclusive (the default) or parallel_read_only.
• Two live runs that share any key never overlap unless both are parallel_read_only. The check happens before a run id is minted, before the acceptance record is written, before the worker starts — a refused job burns nothing and leaves no lost record behind.
• A refusal is a spoken tool result naming the run in the way: "run 4 (audit the repo) is still running and touches the same resource ('/srv/app'); wait for it, stop it, or re-delegate without that key." Never a hang, never a silent queue. check_work reads out what each running job is holding.
• parallel_read_only is believed only when you say so. The declaration is the delegating model's own claim about work it has not done yet — policy input, not a sandbox — so by default it is downgraded to exclusive and recorded that way. TALK_TRUST_DECLARED_READ_ONLY=true lets read-only jobs on a shared key run together; the knob is read at admission time, so turning it back off closes every overlap it had allowed. It is the only thing that can widen behavior.
• No keys means no fence, in either direction: a task that names nothing is exactly the task Talk always ran, record and all.

The fence is per process and covers the api-server and detached lanes, whose runs this registry owns. Inside /talk, the host's own delegation registry runs the child: the job is still checked against the keys this registry holds — never started on top of one — but it holds none itself afterwards, and its WORK_STARTED receipt says so.

Redirecting work that's already running

Say "tell that audit to focus on the token refresh instead" and steer_agent queues the note into the running agent. Steering is not stopping: the agent sees the note after its current step, and the current step always finishes.

The honest part — and the reason this surface looks the way it does — is that the host's steer primitive is a queue write. Queued is not delivered. So every note gets a receipt with a state the substrate can actually prove:

Ask check_work and you hear the note's state in those words — never "they got it" unless the artifact that proves it exists. Since v0.6 every note travels as [tk-xxxxxxxx] note — the token is what the drain preview is matched on, so two agents holding identical text can never land each other's receipts. If a truncated sibling receipt has no exact agent reference, it stays queued/unconfirmed: receipt order and a reused public subagent id never let it inherit another receipt's generation. One substrate note: watching the pre-API drain lowers the host's agent.conversation_loop logger to DEBUG (with a gate filter so operator log output is unchanged) — any DEBUG-guarded computation in that one module becomes active, a bounded perf cost traded for killing the false-"unconfirmed" class. And you often don't have to ask: the host's subagent_stop hook announces a finished background agent into the live call the moment it lands. Those hook events are filtered to the parent session that owns the call; foreign or ownership-less completions are never spoken.

Four tools carry the surface, discovery-first:

• list_agents — everything running, tagged can steer (live subagent ids) or stop only (run numbers). The model resolves "the research one" here, against ids that exist right now.
• steer_agent — subagent ids only. Prefers the host's public steer_subagent (hermes-agent#76805) when present; otherwise resolves the same delegation registry directly and calls the public AIAgent.steer(). A genuine host error is spoken, never routed around.
• redirect_agent — the stronger correction, for "stop, wrong repo": the host's public AIAgent.redirect() (0.20+) aborts the agent's in-flight thinking and retries with your correction, instead of waiting for the next tool boundary. Mid-tool it degrades to the steer queue and says so; on a pre-0.20 host it falls back to steer_agent entirely. Never cancels the work.
• stop_work — the one verb every lane supports: subagents via the host's interrupt_subagent(), api-server runs via POST /v1/runs/{id}/stop, detached one-shots via their retained process handle. Every "want me to stop it?" the refusals offer is backed by this tool — no offered action is fictional.

Runs on the api-server and detached lanes cannot be steered at all — those lanes have no inbound channel — and the refusal says exactly that, then offers the stop that actually works.

Runs are tracked in $HERMES_HOME/state/talk-runs.jsonl. The work is detached, so ending the call does not stop it — but the watcher that would have spoken the result dies with the session, so a run from a previous session is reported as lost, never as "still running".

What the voice can do — the capability bridge

The session prompt carries a bounded, live-catalog section: how many skills are installed, which tool categories are usable right now, and the two rules that keep the model honest — it can delegate anything Hermes can do, and it must never invent tool names. The section is assembled from the real catalog lanes (the host's own registries in-process, or the api-server's /v1/skills + /v1/toolsets + /v1/capabilities out of process); when the catalog is unreachable the section is simply absent, and the session runs on the plain preamble exactly as before.

That changes three everyday exchanges:

• "What can you do?" is answered from the catalog — live evidence, never a recited prompt. A toolset whose tools all failed the host's availability gates is not claimed.
• "Check my screen" / anything past the advertised tools delegates by default. The classification table in talk_operator_auth decides what a host tool call may do at the voice surface: a short curated read-only list (web_search, web_extract, vision_analyze, session_search) runs inline; computer_use's read actions (capture, wait, list_apps, list_windows) need a fresh spoken operator permit; everything else — and every destructive computer-use action, whose in-handler gate fails open without a real approval context — delegates. The classification applies on every transport, the local terminal lane included: mutating host tools steer to delegation with spoken approvals rather than running bare. A denied call never refuses flat: you hear "I can't do that directly in a voice call — I can spin up an agent that can. Want me to?"
• Delegated work that hits a gated action now asks you out loud. The run lane streams the host's approval.request events; Talk speaks the request ("run 3 wants to run a shell command — once, this session, or no?") and your spoken answer resolves it via the new resolve_approval tool, which on Discord rides the same fresh-speech permit machinery as the other mutating tools. Voice can grant once, session, or deny — never always: the choice set is narrowed in code, in the tool schema, and in the prompt, and session is scoped to that run. Fail closed on everything ambiguous: an unanswered question times out into a deny (TALK_APPROVAL_PROMPT_TIMEOUT_S, default 60s), and interrupting the question denies it too — a question not fully heard is not a question answered. Progress and the result still arrive through the existing milestone/result machinery.

TALK_AGENT_PROFILE — which profile the background agent runs under

If your model config lives in a profile rather than the root config.yaml, a bare hermes -z cannot resolve a model and dies with Invalid length for parameter modelId, value: 0. Talk handles this for you:

• TALK_AGENT_PROFILE=<name> — spawn hermes --profile <name> -z ….
• Unset (default): auto-detect. If the root config.yaml names a model.default, no flag is added. If it doesn't and exactly one profile under $HERMES_HOME/profiles/ does, that profile is used.
• Zero matching profiles, or two or more → no flag, deliberately. Guessing between profiles would be invisible until the wrong agent had already run; the spawn's own error names the problem better.
• Set-but-blank (TALK_AGENT_PROFILE=) is an explicit opt out: never pass a flag, even if detection would have found one.

Knobs

The common ones. Every variable — api-server probe internals included — with defaults and failure modes: docs/OPERATING.md.

TALK_IDENTITY_INCLUDE — what the session starts knowing

Three sections are resolved at session start, each independently and each optional:

• PERSONA — your SOUL.md, read through Hermes's own loader (so it gets the same injection scan the text agent's copy does).
• MEMORY — the system-prompt block your configured memory provider contributes. Inside /talk this is the live agent's already-assembled block; standalone, Talk loads the configured provider itself, reads the block, and shuts it down again.
• WORKING — memories/WORKING.md, the one identity file you write rather than the model: who you are, which repos and plugins you mean by name, what an alias maps to. Entries are separated by \n§\n, the same delimiter Hermes uses for MEMORY.md, and each is threat-scanned independently — one bad entry costs that entry, not your whole table. When a host is attached, one sentence is appended pointing at search_memory for names not in the file. The rule to ASK when a spoken name could match more than one thing rides the voice preamble itself, on every lane — it depends on no file, no tool, and no include list, so nothing can drop it.

A broken or missing provider costs that section and nothing else — the call still starts. talk_status reports which sections resolved and how many characters each contributes, never their content.

WORKING.md is what stops a voice session asking who you are every call. Nothing fills it for you — no producer writes installed plugins or recent work into it; what you curate by hand is all a session gets. It is read once at session mint and stays frozen for the call: an edit lands on the NEXT session, never the live one. Keep it short — the resolved prompt is resident and paid for on every turn:

Pedro, solo operator. Ships at night, prefers blunt answers.
§
"Talk" or "hermes-talk" means TheSmokeDev/hermes-talk (this plugin).
§
"Dograh" (often heard as "Dobra" or "Dog Bras") is the voice stack.

Conflicting entries are left alone on purpose. Two lines claiming the same alias both travel, because resolving that by file order would silently bind your words to whichever line you happened to write first — the model is told to ask instead.

Set TALK_IDENTITY_INCLUDE=MEMORY,PERSONA to pin the list. The trap: this REPLACES the default rather than extending it — TALK_IDENTITY_INCLUDE=MEMORY means memory and nothing else, and the only symptom is a session that has quietly stopped knowing who it's talking to. Unknown names are dropped silently, so a typo narrows the prompt instead of taking voice down. The upgrade trap is the same trap, aged: a list pinned before WORKING existed (e.g. MEMORY,PERSONA) keeps working verbatim and silently drops your curated context after upgrading — the session logs one warning at mint when a pinned list lacks WORKING.

Sections are capped (PERSONA 4,000 chars, MEMORY 6,000, WORKING 2,000). A Realtime session's instructions are resident for the whole call and paid on every turn, so these are a budget, not a nicety. Caps trim from the tail, and each section puts what is KNOWN before what can be looked up — so an oversized file loses its lookup pointer before it loses your facts.

Design rules

The three that shaped everything else:

• Nothing fails quietly. A degraded backend, a missing tool, a run whose watcher died — each is said out loud in the conversation. A voice surface that silently does less than you asked is worse than one that refuses.
• The credential never leaves the process. Key or OAuth token hits exactly one OpenAI endpoint (the mint) and the socket only ever sees the ephemeral secret it returns.
• Hermes owns the tools and the session. The Realtime layer is ears, mouth, and turn-taking. It never owns the agent loop.

Current boundaries

Stated here so nobody has to discover them on a call:

• No self-hosted realtime lane. All three providers are hosted. The provider contract (talk_realtime.py) is where a local model plugs in, and the core registration already declares capabilities per lane — so a local lane can ship honestly as talk-without-tools the day a model can carry it. Today none is shipped.
• The dashboard tab is OpenAI-only, and the cascade voice is OpenAI-only. Grok has no WebRTC offer endpoint; only OpenAI's text-output mode is wired and verified for the cascade.
• Discord refuses Gemini. The gated-response authorization flow has no Live wire equivalent, so connect fails closed rather than answering unvetted speakers.
• Memory writeback covers terminal and Discord, not the dashboard tab (its Realtime events stay in the browser).
• Result delivery is bound to the session that started the work. Same Hermes session reconnecting: spoken exactly once. A different session, or a run from a previous process: reported lost, never "still running".
• Steering reaches subagents only. api-server and detached runs are stop-only; the refusal says so and offers the stop that works.
• Open mic with server-side turn detection. No wake word, no push-to-talk. TALK_SESSION_KEY is a shared memory scope, not a speaker boundary — leave it unset in multi-user channels.
• No latency instrumentation. The plugin emits no first-audio metric; the timings quoted in the cascade section are not something it measures for you on a call.

Where this sits upstream

• RFC NousResearch/hermes-agent#77111 — filed from this repo: a RealtimeVoiceProvider ABC for Hermes core, because four open PRs were building duplex voice independently and the category deserves an interface rather than a merge queue. This plugin is the working reference implementation for that discussion, not a bid to be merged.
• PR NousResearch/hermes-agent#101808 — the core contract, orchestrator, and first built-in provider, ported from this plugin's orchestrator and OpenAI transport. hermes-talk already publishes its three lanes on that contract (see Hermes core realtime contract).
• PR NousResearch/hermes-agent#97325 — a pointer to this plugin on the official Voice Mode docs page.

Background — why this exists

OpenAI shipped this exact pattern for Codex on 2026-07-23 — voice as a control layer over concurrent agents. It's excellent, and it's closed: paid ChatGPT plans only, and GPT-Live has no developer API. Their own docs point builders back at the Realtime API. So that's what this is built on, for an agent you actually own.

Status

Under active development; the PyPI badge above is the released version. Every version with its receipts: CHANGELOG.md. Open epics and threads: #19 provider-neutral Realtime voice platform, #32 operator-grade orchestration UX, #43 Talk over the Bot Mode roster, #44 channel-neutral voice transport.

Contributing

uv sync --extra dev (or pip install -e ".[dev]"), pytest -q, ruff check . — offline, no keys, seconds. Priorities, the path for each kind of change (a provider, a surface, a tool, a fix), the merge bar, and the one test trap on a box that has Hermes installed: CONTRIBUTING.md. First response within 24 hours; good first issue fits in one sitting; a provider that works or broke for you is a contribution too (docs/PROVIDER-RECEIPT.md).

Contributors adapting The Homie's v1.7.0 capability-plugin lessons to Hermes should use the capability-kernel port plan. It maps the reusable safety and lifecycle contracts onto Hermes-owned APIs; it does not claim that hot lifecycle support already exists here.

Contributors

@danclaw93: room-scoped spoken approvals send the request_id the Hermes run API reads, so they stop failing with HTTP 400 (0.17.1).

@kvnloo: PulseAudio WebRTC echo cancellation on Linux, and the fix that stopped quiet words being clipped during playback (#81); semantic turn-detection controls across the three lanes (#107, in review). @TheAngryPit — a renderer-owned Realtime transport for the Hermes desktop app that keeps core as the single chat authority (#80, in review). @webdevtodayjason — field-tested feedback from a second live consumer on the upstream RealtimeVoiceProvider contract these lanes register on (hermes-agent#81404).

License

MIT. Adapted-source licenses and contributor credits are in THIRD_PARTY_NOTICES.md.