Hermes Talk

Realtime duplex voice for your own agent — talk to it, it runs real work in the background, it reports back out loud.

Hermes Talk is a realtime voice plugin for Hermes Agent: you speak, it answers out loud, and it calls the agent's own tools without leaving the conversation. It runs in the terminal (hermes talk), in a Discord voice channel (/talk join), and in the Hermes dashboard Talk tab.

It rides your ChatGPT or SuperGrok subscription. No API key required.

The realtime lanes are OpenAI Realtime (gpt-realtime-2.1, on your ChatGPT subscription through codex login), xAI Grok Voice (on an X Premium or SuperGrok login, no key), and Gemini Live (which does need a GEMINI_API_KEY — free-tier AI Studio keys work); GPT-Live is selected separately by TALK_VOICE_MODE=live and runs gpt-live-1-codex on the Codex subscription or gpt-live-1 on explicitly chosen API billing.

Talk calls the host's tools, delegates background work while you keep talking, reports results, and handles current approval requests. Provider support differs by surface; see the tables below.

What it is: a plug-in that adds interruptible, duplex speech-to-speech voice to an existing Hermes Agent install — one bidirectional audio session in which the agent uses its real tools, hands work to background agents, and speaks the results when they land.

Who it is for: people already running Hermes Agent who would rather talk to it than type at it, and who want to keep talking while it works. It is not a standalone assistant, and it does not replace Hermes's built-in turn-based voice mode — that is a different shape, and a good one.

🔊 Watch the 2:27 cut with sound — delegate, keep talking, hear the result land.

Real Realtime session, 8× speed in the GIF, recorded at v0.3.0. It does not demonstrate the new GPT-Live or Codex-worker integration.

Install

hermes plugins install TheSmokeDev/hermes-talk --enable
pip install "hermes-talk[audio]"   # mic + speaker support (sounddevice); skip if dashboard-only

Needs Python ≥ 3.11 and a Hermes host ≥ v0.17. GPT-Live, shared task attachment and Codex workers need additional host capabilities; the full list is in Prerequisites.

Quickstart

Terminal — start here:

hermes talk

You are live: speak, and it answers out loud in the same breath. Ctrl+C hangs up. → Use

Discord — the call happens inside a voice channel, not in chat:

/voice join     # put Hermes in the voice channel first
/talk join      # Talk borrows that connection — it never opens a second one

Talk answers in the room everyone can hear; /talk leave ends it. Mutating tools stay denied until you set TALK_DISCORD_OPERATOR_USER_IDS. → Discord voice

Dashboard — browser audio, no local mic drivers:

hermes dashboard    # then open the Talk tab and hit Start

Allow the microphone and talk. You see the live transcript plus a list of background runs. → Dashboard tab

New in 0.20.0

• A floating Talk panel shared by the dashboard tab and the Desktop Talk view, with a runtime that survives collapsing the controls and browsing other tasks.
• POST /text/input and a textInput descriptor on GET /status: one authenticated route for explicit operations from the panel.
• Live replay — POST /live/speech with replay:true re-announces a terminal result into the exact bound Live session.
• POST /native/attach accepts input_mode:"typed" for microphone-off use that mints no voice credentials, plus read-only recipient catalog, history, status and selection routes.
• Result presentation across transports: result ready, context submitted, playback started, playback finished, interrupted and unknown stay separate facts; native terminal and Discord gain /replay EVENT_ID.
• Talk inside the current Desktop conversation, plug-and-play: open a conversation, Talk, Connect.

Every version with its receipts: CHANGELOG.md.

Providers and billing

The provider knob is fail-closed and never inferred from which keys exist. Per-lane detail and the credential order: docs/PROVIDERS.md. Speaking in a voice of your own: docs/CASCADE.md.

Surfaces

Is it working?

hermes plugins list        # → hermes-talk · enabled · current version
hermes talk doctor         # → read-only: auth lane, provider, model/voice, audio, host lanes
hermes talk check          # → doctor + one live provider turn + one bounded Hermes run
# then, in any session: say "status report" — talk_status answers with
# version, auth lane, agent lane, and audio state.

Doctor is read-only by design: it names which lane came up and what is missing, and never writes, probes, or refreshes a token. check is the other half and is deliberately not read-only — one short provider turn and one short agent run, exit 0 only if every step passed. A mock can never go green.

Filing an issue? hermes talk diagnostics --bundle writes one redacted, owner-only file — versions, the names of the variables you have set, device and host facts, and every doctor outcome; no values, logs, prompts, transcripts, audio, or paths. It is safe to paste into a public issue and it is what the bug template asks for.

Upgrade with hermes plugins update hermes-talk — not a second install — then restart the gateway: a running process keeps executing the old code until you do.

The full diagnostic walk, every receipt, and the upgrade runbook: docs/OPERATING.md.

Documentation

Everything Hermes Talk does, in depth:

• OPERATING.md — install, upgrade, use, every knob, the three agent lanes, the Discord lane, the dashboard tab, current boundaries, troubleshooting.
• PROVIDERS.md — per-lane provider detail and the fail-closed OpenAI credential order.
• BACKGROUND-WORK.md — delegation, admission control, steering a running agent, and the capability bridge.
• CASCADE.md — the cascade lane: your own ElevenLabs voice over a realtime provider.
• GPT-LIVE.md — GPT-Live billing and voice, task attachment, Codex workers, operator acceptance.
• DESKTOP.md — Talk in the Hermes desktop app: the stock lane, and what the Talk-enabled build adds.
• VOICE-COMMANDS.md — say this, hear this, and what each spoken receipt commits to.
• REALTIME-ORCHESTRATOR.md — architecture map of the tool-calling realtime lane.
• dashboard-task-continuity.md — joining, continuing and reconnecting to a canonical Hermes task.
• recipient-routing.md — addressing an existing application task from a voice task.
• codex-workers.md — selecting a Codex background worker from a bound task.
• task-event-projection.md — the worker-side library that restores task and work state.
• passive-attachment-client.md — the shared passive-history client used by all three surfaces.
• PROVIDER-RECEIPT.md — how to report a provider lane that worked, or broke, for you.
• CAPABILITY-KERNEL-PORT.md — the capability-plugin kernel adaptation guide.

Design rules

The three that shaped everything else:

• Nothing fails quietly. A degraded backend, a missing tool, a run whose watcher died — each is said out loud in the conversation. A voice surface that silently does less than you asked is worse than one that refuses.
• The credential never leaves the process. Key or OAuth token hits exactly one OpenAI endpoint (the mint) and the socket only ever sees the ephemeral secret it returns.
• Hermes owns the tools and the session. The Realtime layer is ears, mouth, and turn-taking. It never owns the agent loop.

Background

Hermes Talk began as a plugin and became a reference implementation for the speech-to-speech contract Hermes core now carries:

• RFC NousResearch/hermes-agent#77111 — filed from this repo: a RealtimeVoiceProvider ABC for Hermes core.
• PR NousResearch/hermes-agent#101808 — the core contract, orchestrator, and first built-in provider, ported from this plugin's orchestrator and OpenAI transport. hermes-talk already publishes its three lanes on that contract (details).
• PR NousResearch/hermes-agent#97325 — a pointer to this plugin on the official Voice Mode docs page.

Status

Under active development; the PyPI badge above is the released version. Every version with its receipts: CHANGELOG.md. Open epics and threads: #19 provider-neutral Realtime voice platform, #32 operator-grade orchestration UX, #43 Talk over the Bot Mode roster, #44 channel-neutral voice transport.

Contributing

uv sync --extra dev (or pip install -e ".[dev]"), pytest -q, ruff check . — offline, no keys, seconds. Priorities, the path for each kind of change (a provider, a surface, a tool, a fix), the merge bar, and the one test trap on a box that has Hermes installed: CONTRIBUTING.md. First response within 24 hours; good first issue fits in one sitting; a provider that works or broke for you is a contribution too (docs/PROVIDER-RECEIPT.md).

Contributors adapting The Homie's v1.7.0 capability-plugin lessons to Hermes should use the capability-kernel port plan. It maps the reusable safety and lifecycle contracts onto Hermes-owned APIs; it does not claim that hot lifecycle support already exists here.

Contributors

@danclaw93: room-scoped spoken approvals send the request_id the Hermes run API reads, so they stop failing with HTTP 400 (0.17.1).

@HearthCore: diagnosed why Desktop Talk could never connect on a stock Hermes host, and made it connect there; the composer popover also stops closing when the pointer crosses the chat (#169, #170, 0.21.1).

@kvnloo: PulseAudio WebRTC echo cancellation on Linux, and the fix that stopped quiet words being clipped during playback (#81); semantic turn-detection controls across the three lanes (#107, in review). @TheAngryPit — a renderer-owned Realtime transport for the Hermes desktop app that keeps core as the single chat authority (#80, in review). @webdevtodayjason — field-tested feedback from a second live consumer on the upstream RealtimeVoiceProvider contract these lanes register on (hermes-agent#81404).

License

MIT. Adapted-source licenses and contributor credits are in THIRD_PARTY_NOTICES.md.