Hermes Talk
Realtime voice for Hermes Agent. Press Talk, say what you need, and cut in whenever you like. Your agent keeps using its real tools while you talk.
Hermes Talk is a plugin for Hermes Agent that turns it into something you can talk to. You speak, it answers out loud, and you can interrupt it mid-sentence. While you talk it calls the agent's own tools, hands longer jobs to background agents, and tells you how they went when they finish.
It runs in the terminal (hermes talk), in a Discord voice channel (/talk join),
in the Hermes dashboard Talk tab, and in Hermes Desktop.
You probably don't need a new API key. OpenAI Realtime runs on your ChatGPT
subscription through codex login, and Grok Voice runs on an X Premium or
SuperGrok login. Gemini Live needs a GEMINI_API_KEY, and free-tier AI Studio
keys work. If you'd rather keep it all at home, the local lane talks to your
own model server.
Hermes Talk adds duplex voice to a Hermes install you already run. It is not a standalone assistant, and it doesn't replace Hermes's built-in turn-based voice mode, which is a different (and good) design.
Watch the 2:27 cut with sound: delegate a job, keep talking, hear the result land. It is a real Realtime session recorded at v0.3.0 (the GIF runs at 8x), so it predates GPT-Live, the Codex workers and the 0.24.0 panel.
New in 0.24.0
A big community release. The community features in the new Experimental section ship switched off, so the Talk button you know works exactly as before.
- One-button Talk. Desktop and the dashboard Talk tab now open on one large voice button with a short status line. Pressing it starts Talk in one step, exactly as before. Mute and Sleep show up beside it while you're connected. Typing, recipients and attachments sit under Type a message, and voice and conversation choices live under Settings.
- Experimental settings. A new Experimental section in the Hermes Talk plugin settings holds a saved Amazon Echo, a text-only call journal, voice language and a speaker echo hold for speakers with output lag, like a TV over HDMI. Echo and the journal start off, language starts on Auto, and the echo hold starts at 0. Details: Experimental plugin settings.
- Switch chats by voice. On a compatible Hermes host you can say "Switch to my Codex chat and read its last reply", then "Ask it to check mobile too." It works with Claude Code chats as well. If several chats match, Talk asks which one you mean. Host requirements and the full phrase list are in recipient-routing.md.
- Twilio phone server (experimental).
python -m talk_twilio_serveranswers calls to your own Twilio number with a fresh Gemini Live session that has no task and no tools. Webhooks are signature-checked, callers must be on an allowlist, and normal Talk never loads it. It hasn't been confirmed on a real call yet. See TWILIO.md. - Profile isolation. On a multiplexed gateway, Talk settings and keys come from the profile serving the turn, Discord voice uses that profile's own bot, and background work keeps the profile that started it.
- Discord playback and honest outcomes. Replies stop cleanly at their boundaries and queued announcements resume after playback drains. A run that failed, was stopped or was interrupted is now announced as exactly that. Delegated work also stops filling the call with empty "still working" lines.
- Fixes people asked for. The end-of-call transcript review no longer
re-spawns itself. Gemini Live no longer fails to start with close code 1007
on tool schemas.
hermes talk doctorno longer crashes on a host with no audio system. - One sign-in for the dashboard. When the dashboard's own sign-in is on,
that sign-in opens the Talk tab, with nothing to paste per browser tab. An
explicit
TALK_DASHBOARD_TOKENstill protects deployments that set one.
Still fresh from 0.23: an optional Phone mode
powered by Dograh, and TALK_PROVIDER=local for running the voice lane on your
own GPU (LOCAL.md). From 0.22: start, check and steer Desktop
sessions by voice (DESKTOP.md).
Every version and what it shipped: CHANGELOG.md.
Install
hermes plugins install --ref 012c1069d725decce26aae480217093f647776bf TheSmokeDev/hermes-talk --enable
pip install "hermes-talk[audio]" # mic + speaker support (sounddevice); skip if dashboard-only
You need Python 3.11 or newer and a Hermes host at v0.23.0 or newer. GPT-Live, shared task attachment and Codex workers need extra host capabilities, listed in Prerequisites.
That commit is the published v0.24.0 source. Hermes v0.22.5 wants a full
40-character commit SHA for --ref and won't take a tag, and pinning the SHA
also avoids an older community-catalog pin. To move an existing pinned install,
keep any local plugin edits safe, then run:
hermes plugins install --force --ref 012c1069d725decce26aae480217093f647776bf --enable TheSmokeDev/hermes-talk
Restart the gateway, the dashboard and Desktop afterwards. A running process
keeps the old code until you do. An unpinned Git install can use
hermes plugins update hermes-talk.
Using Desktop? Installed agent packages are opt-in there. In Desktop Capabilities → Plugins, turn on the Desktop half of Hermes Talk if it is off. Installing the Python wheel alone doesn't register it; the plugin installer does. More in the Desktop guide.
Quickstart
Terminal. Start here:
hermes talk
Speak, and it answers out loud. Ctrl+C hangs up. More in Use.
Discord. The call happens in a voice channel, not in chat:
/voice join # put Hermes in the voice channel first
/talk join # Talk borrows that connection; it never opens a second one
Everyone in the room hears the answers, and /talk leave ends the call.
Tools that change things stay denied until you set
TALK_DISCORD_OPERATOR_USER_IDS. See
Discord voice.
Dashboard. Browser audio, no local mic drivers:
hermes dashboard # then open the Talk tab and press the voice button
Allow the microphone and talk. You get the live transcript and a list of background runs. See Dashboard tab.
Desktop. Open a conversation, click Talk in the top bar or beside the composer, then press Start talking. The popover closes once you're connected and the audio keeps running; Stop stays beside the composer. On stock Desktop, send one message first so the conversation exists. See DESKTOP.md.
Providers and billing
| Lane | How it authenticates | Default model | Surfaces |
|---|---|---|---|
OpenAI Realtime (TALK_PROVIDER=openai, default) |
ChatGPT subscription through codex login, or TALK_OPENAI_API_KEY / OPENAI_API_KEY |
gpt-realtime-2.1 |
every surface |
GPT-Live (TALK_VOICE_MODE=live) |
TALK_LIVE_AUTH=subscription (the default, on the Codex subscription) or explicitly chosen api billing; no automatic paid fallback |
gpt-live-1-codex (subscription) / gpt-live-1 (API) |
terminal, Discord, dashboard, Desktop, on an explicit task |
xAI Grok Voice (TALK_PROVIDER=grok) |
an X Premium or SuperGrok login (hermes auth add xai-oauth), no API key, or TALK_XAI_API_KEY / XAI_API_KEY |
grok-voice-latest |
terminal + Discord |
Gemini Live (TALK_PROVIDER=gemini) |
GEMINI_API_KEY / TALK_GEMINI_API_KEY; free-tier AI Studio keys work |
gemini-3.1-flash-live-preview |
terminal; Discord refuses it for now |
Local models (TALK_PROVIDER=local) |
none by default; any self-hosted OpenAI-Realtime-compatible server, e.g. speech-to-speech with an Ollama LLM | the server's own | terminal (Discord not yet verified) |
Cascade voice (TALK_VOICE_MODE=cascade) |
TALK_ELEVENLABS_API_KEY / ELEVENLABS_API_KEY, on top of the OpenAI Realtime lane |
eleven_flash_v2_5 |
terminal, Discord, dashboard |
Talk never guesses the provider from whichever keys happen to be set. An unknown
value refuses to start. Grok supports all 28 documented built-in voices through
TALK_GROK_VOICE (for example, carina).
- Per-lane detail, the voice lists and the credential order: PROVIDERS.md
- Fully local on your own GPU: LOCAL.md
- Your own ElevenLabs voice: CASCADE.md
- GPT-Live billing, voices and task attachment: GPT-LIVE.md
- Experimental xAI speech effects and phrase wrappers: XAI-SPEECH-TAGS.md
Surfaces
| Surface | Entry point |
|---|---|
| Terminal | hermes talk; attach to a task with hermes talk --task TARGET |
| Inside a Hermes session | /talk, the one surface with an attached agent loop, so lookups and delegation answer inline |
| Discord voice channel | /voice join, then /talk join [TARGET] |
| Dashboard Talk tab | hermes dashboard, open the tab, press the voice button |
| Desktop Talk button | open a conversation, Talk, Start talking; works on stock Desktop, with a floating window on the Talk-enabled build (host lanes) |
| Amazon Echo (experimental) | hermes talk echo HOST with echo-dot-assist's hassmic in Wyoming mode; each wake word starts a session. Not yet confirmed on physical Echo hardware (setup) |
Optional native Phone mode
Phone mode lets you continue a Hermes conversation over a phone call, with its existing history, memory and tools. It is part of Hermes Talk, powered by Dograh, and off by default. Normal Talk needs no Dograh account.
To set it up, enable it in Hermes's plugin settings, then connect a dedicated Dograh account and a caller number you already own:
hermes talk phone setup # connect Dograh and your number
hermes talk phone doctor # connection checks
Call tools, review controls, status and caller questions all run through the
Hermes conversation you already have. /phone review CALL_ID opens the host's
native approval picker, and /phone answer REQUEST_ID ANSWER sends an
authenticated owner answer back into the call. Desktop, dashboard and mobile
viewers use the host's message stream; Discord uses its existing adapter and
channel or thread. There is no separate Phone dashboard.
Some ground rules:
- Installation never dials or buys a number.
- Dograh and its configured carrier, STT, LLM and TTS services can charge for carrier use, hosting and models. Audio, transcripts and recordings are handled according to that account's provider settings.
- Keep keys in Hermes secret settings, allowlist exact owned numbers, and review every call.
- Phone Stop prevents new calls and revokes grants. It does not hang up a call the carrier already has in progress.
- The optional in-call bridge starts only with
hermes talk phone bridge. Configure its private relay or HTTPS proxy before you accept remote caller questions. - An existing separate Dograh test setup can be imported with
hermes talk phone setup --import-dograh. The original journal is kept and its old approvals are revoked.
The Twilio phone server is a different, standalone experiment and is not part of Phone mode.
Is it working?
hermes plugins list # hermes-talk, enabled, current version
hermes talk doctor # read-only: auth lane, provider, model/voice, audio, host lanes
hermes talk check # doctor plus one live provider turn and one bounded Hermes run
Then, in any session, say "status report". talk_status answers with the
version, auth lane, agent lane and audio state.
doctor only reads. It names the lane that came up and what is missing, and it
never writes, probes or refreshes a token. check is deliberately not read-only:
it runs one short provider turn and one short agent run, and exits 0 only if
every step passed. A mock can't turn it green.
Filing an issue? hermes talk diagnostics --bundle writes one redacted,
owner-only file with versions, the names of the variables you've set, device
and host facts, and every doctor outcome. It holds no values, logs, prompts,
transcripts, audio or paths, so it is safe to paste into a public issue, and it
is what the bug template asks for.
The full diagnostic walk and the upgrade runbook: OPERATING.md.
Documentation
Start with these:
- OPERATING.md: install, upgrade, use, every setting, the agent lanes, Discord, the dashboard tab, current limits and troubleshooting.
- DESKTOP.md: Talk in Hermes Desktop, the stock lane, and what the Talk-enabled build adds.
- PROVIDERS.md: each provider lane and the OpenAI credential order.
- VOICE-COMMANDS.md: what to say, what you'll hear back, and what each spoken receipt promises.
- BACKGROUND-WORK.md: delegation, admission control, steering a running agent, and the capability bridge.
Going deeper:
- GPT-LIVE.md, CASCADE.md, LOCAL.md, TWILIO.md, XAI-SPEECH-TAGS.md
- recipient-routing.md: addressing an existing Codex or Claude Code chat from a voice task.
- codex-workers.md: picking a Codex background worker from a bound task.
- dashboard-task-continuity.md: joining, continuing and reconnecting to a Hermes task.
- REALTIME-ORCHESTRATOR.md: how the tool-calling realtime lane is built.
- task-event-projection.md and passive-attachment-client.md: the shared task-state and passive-history libraries.
- PROVIDER-RECEIPT.md: report a provider lane that worked, or broke, for you.
- CAPABILITY-KERNEL-PORT.md: adapting The Homie's capability-plugin lessons to Hermes. It maps the contracts; it does not claim hot lifecycle support exists here.
Design rules
Three rules shaped everything else.
- Nothing fails quietly. A degraded backend, a missing tool or a run whose watcher died gets said out loud in the conversation. A voice surface that quietly does less than you asked is worse than one that refuses.
- The credential never leaves the process. The key or OAuth token reaches exactly one OpenAI endpoint (the mint), and the socket only ever sees the short-lived secret that endpoint returns.
- Hermes owns the tools and the session. The Realtime layer handles ears, mouth and turn-taking. It never owns the agent loop.
Background
Hermes Talk started as a plugin and became a reference implementation for a proposed speech-to-speech contract in Hermes core:
- RFC NousResearch/hermes-agent#77111,
filed from this repo, proposes a
RealtimeVoiceProviderABC for Hermes core. - PR NousResearch/hermes-agent#101808 carried the proposed core contract, orchestrator and first built-in provider, ported from this plugin. It was closed without merging, so no Hermes release has the contract today. Hermes Talk detects it and publishes its three lanes only on a host that has it (details).
- PR NousResearch/hermes-agent#97325 adds a pointer to this plugin on the official Voice Mode docs page.
Open epics: #19 provider-neutral Realtime voice platform, #32 operator-grade orchestration UX, #43 Talk over the Bot Mode roster, and #44 channel-neutral voice transport.
Built with the community
Thanks to everyone who contributed code and reported issues. Here is who did what, from the CHANGELOG credit lines and merged pull requests.
Code and features
- @Philibergus: the call journal, the spoken-language settings, one sign-in for the dashboard Talk tab, and the fix that brought back its token field (0.24.0).
- @tancou: profile context for background work, Discord playback completion, honest run outcomes (0.24.0), the CI race fix (0.23.1), and OpenAI-style list envelopes in REST discovery (0.23.0).
- @Akm0d: per-profile settings, keys and Discord bot on multiplexed gateways (0.24.0).
- Kiven (@abaichuan1017): a bounded, shared capability-catalog probe, so a stuck registry no longer pins the caller (0.24.0).
- @computator1200: the experimental Twilio phone server (0.24.0).
- @patrickhaahr: the Amazon Echo surface (0.24.0).
- @zoidypuh: Grok's expanded built-in voice list and the xAI speech-tag reference (0.22.0).
- @gabrieljudah: the presence bus that lets another surface mirror or drive a call (0.22.0, #171).
- @HearthCore: worked out why Desktop Talk could never connect on a stock Hermes host, and made it connect (#170, 0.21.1).
- @dbdefault: isolated installed Hermes registries in the catalog tests (#195).
- @vivekrathod: routed offloaded Talk commands back to the gateway loop (#187).
- @teknium1: the Codex lane stopped rewriting
~/.codex/auth.jsonand borrows Hermes's Codex login first (#146). - @danclaw93: room-scoped spoken approvals stopped failing with HTTP 400, the first outside fix to a live lane (0.17.1).
- @JakeStevenson: made dashboard teardown survive a failing step, found while building a mobile surface on its transport (0.17.3).
- @kvnloo: PulseAudio echo cancellation on Linux and the fix that stopped quiet words being clipped during playback (#81).
- @TheAngryPit: a renderer-owned Realtime transport for Hermes Desktop (#80, in review).
- @webdevtodayjason: field-tested feedback on the upstream
RealtimeVoiceProvidercontract (hermes-agent#81404).
Bug reports
@gocoelho (#227, #228, #229), @marcablanca (#236), @rmccaincns (#239), @JoshCollis (#202), @kvnloo (#215), @Neuraladvance (#214), @Sterling-Automation (#198), @blacksheep6a (#184, #185), @izzyjade06 (#180) and @eabase (#122).
Come build with us
New features land behind Experimental settings, off by default, so you can ship something ambitious without changing anyone's Talk button. Getting started takes seconds and needs no keys:
pip install -e ".[dev]" -r requirements-lint.txt
pytest -q
ruff check .
CONTRIBUTING.md covers priorities, where each kind of change
starts (a provider, a surface, a tool, a fix), the merge bar, and the one test
trap on a machine that has Hermes installed. You'll get a first response within
24 hours, and merged work is credited by name in the CHANGELOG
and here. A good first issue
fits in one sitting, and a provider receipt for a
lane that worked or broke for you counts too. Ideas and questions go to
Discussions.
License
MIT. Adapted-source licenses and contributor credits are in THIRD_PARTY_NOTICES.md.
Metadata
Release files for hermes-talk 0.24.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| hermes_talk-0.24.1.tar.gz | 1.3 MB | With provenance | Details for hermes_talk-0.24.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| hermes_talk-0.24.1-py3-none-any.whl | Python 3 | none | any | Details for hermes_talk-0.24.1-py3-none-any.whl |
Total release size: 1.9 MB
Release files / hermes_talk-0.24.1.tar.gz
| Download URL | hermes_talk-0.24.1.tar.gz |
|---|---|
| Size | 1.3 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f94d84d7b8b6b8b430d0c43265b7da4470e6eb5575e344cc42c5595b874be8df
|
|
BLAKE2b-256 checksum How to use checksums |
249d5978628e7f237bc954f624e5235faa31f78b4cfd2eefc4b5d7b6b72a4b63
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.
Transparency logRelease files / hermes_talk-0.24.1-py3-none-any.whl
| Download URL | hermes_talk-0.24.1-py3-none-any.whl |
|---|---|
| Size | 590.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
e933400bc00da2d055562b63e8aa2bd2e5cc78c5505254da7afd5a0dd2379883
|
|
BLAKE2b-256 checksum How to use checksums |
1fbe9d65ba63fbe604f4fe0e9434271d5bf07fd09dd3df5fd4565a8ab406f16e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.
Transparency logSecurity for hermes-talk 0.24.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for hermes_talk-0.24.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.