Skip to main content

voice_runtime

Provider-agnostic voice call runtime for telephony projects. Manages audio queues, mark synchronization, STT/TTS providers, and transport protocols — so consumers focus on conversation logic, not plumbing.

Explicit regional Twilio REST APIs (0.1.14)

Use hangup_call(..., regions=...) for ordered regional hangup and list_calls(...) for complete, typed, destination-filtered reads. Account path identity is separate from API-key authentication. Regional absence is not proof that a call ended; unknown timestamps and incomplete reads remain explicit.

See the migration guide and independent examples. Existing SMS/outbound behavior is unchanged; the old CDR reader is not silently replaced. This package does not configure consumer secrets or rollout policy.

Quick Example

Make a call, say something, listen for a response via on_committed callback, hang up:

import asyncio
import threading
import time
import uvicorn
from fastapi import FastAPI

from voice_runtime.session import VoiceSession
from voice_runtime.transports.twilio_ws import register_voice_websocket
from voice_runtime.transports.twilio_call import initiate_outbound_call
from voice_runtime.tts import create_tts
from voice_runtime.stt import create_stt

# 1. Create session and start WebSocket server
session = VoiceSession()
app = FastAPI()
register_voice_websocket(app, session)

def run_server():
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    session.set_loop(loop)
    loop.run_until_complete(uvicorn.Server(
        uvicorn.Config(app, host="0.0.0.0", port=8080, log_level="warning")
    ).serve())

threading.Thread(target=run_server, daemon=True).start()
time.sleep(1)  # wait for server to start

# 2. Initiate call (Twilio calls back to our /voice WebSocket)
call_sid = initiate_outbound_call("+358401234567")
session.call_sid = call_sid
session.wait_for_ws_connect(timeout=30)

# 3. Speak
tts = create_tts()  # default: ElevenLabs
tts.speak("Hello! How are you today?", session)
session.send_mark_and_wait("after-greeting")  # block until playback done

# 4. Listen — persistent STT with on_committed callback
transcript_event = threading.Event()
heard = []

def on_committed(text: str):
    heard.append(text)
    transcript_event.set()

session.stt_factory = lambda: create_stt()  # default: ElevenLabs
# Transport starts STT automatically; on_committed fires for each utterance

transcript_event.wait(timeout=30)
print(f"Caller said: {heard}")

# 5. Hang up
session.request_disconnect()

What happens under the hood

Consumer thread              voice_runtime                 Transport (Twilio)
─────────────────────────────────────────────────────────────────────────────
initiate_outbound_call()  →  Twilio REST calls.create()  → Twilio dials phone
                             build_stream_twiml()           with <Connect><Stream>
wait_for_ws_connect()     ←  signal_ws_connected()       ← Twilio opens /voice WS
                                                            stt_factory() → stt.start()
tts.speak(text, session)  →  ffmpeg MP3→μ-law             → send_audio task
                             put_outbound_sync()             sends base64 frames
send_mark_and_wait()      →  get_pending_mark()           → WS sends mark JSON
                          ←  signal_mark_received()       ← WS receives mark echo
on_committed(text)        ←  stt.on_committed()           ← WS receives media frames
                                                             decodes base64 → STT
request_disconnect()      →  _disconnect_requested.set()  → watch_disconnect task
                                                             stt.stop(), closes WS
                                                             Twilio ends call

Architecture

┌──────────────────────────────────────────────┐
│ Consumer (outcaller, ninchat_voice)          │
│  - Subclass VoiceSession (e.g. TelcoSession) │
│  - Call speak(), listen(), hang up           │
├──────────────────────────────────────────────┤
│ voice_runtime                                │
│  - VoiceSession: queues, marks, intents      │
│  - Factories: create_stt / create_tts        │
│  - Providers: ElevenLabs, Azure, Google      │
│  - SttTee: dual-provider fan-out             │
│  - Audio: G.711 μ-law codec + mixer          │
├──────────────────────────────────────────────┤
│ Transport (protocol-specific)                │
│  - twilio_ws: Media Streams WebSocket        │
│  - twilio_call: REST call initiation + TwiML │
└──────────────────────────────────────────────┘

Key invariant: VoiceSession has zero transport or provider imports. Consumers never import Twilio or ElevenLabs directly — they use factories and the intent API.

Factories

Provider-agnostic factories mirror the yamlgraph create_llm() pattern:

from voice_runtime.stt import create_stt, get_stt_class
from voice_runtime.tts import create_tts
from voice_runtime.transport import create_transport

stt = create_stt(provider="elevenlabs")     # or "azure", "google"
tts = create_tts(provider="elevenlabs")     # or "azure", "google"
transport = create_transport(provider="twilio")

# get_stt_class returns the class without instantiating (for factory arguments)
SttClass = get_stt_class(provider="elevenlabs")
session.stt_factory = lambda: SttClass(language_code="en")

Google Chirp 3 STT and TTS (0.5.0, VR-013)

Install the optional extra: pip install "voice-runtime[google]". Without it, nothing Google is imported and the other providers behave exactly as before.

Authentication. Application Default Credentials (ADC) only: workload identity in pods, gcloud auth application-default login locally. The project is also sent as the quota project, which user credentials require. No key is read, stored or logged by the providers.

Configuration. An explicit non-None constructor argument wins over the environment, and the environment wins over the default.

Argument Environment variable Default
project first non-empty of GOOGLE_SPEECH_PROJECT, GOOGLE_CLOUD_PROJECT none: ValueError at STT start() / TTS speak()
location GOOGLE_SPEECH_LOCATION eu (eu-speech.googleapis.com, eu-texttospeech.googleapis.com)
model (STT) GOOGLE_STT_MODEL chirp_3
language_code (STT) GOOGLE_STT_LANGUAGE fi-FI
phrase_hints (STT) GOOGLE_STT_PHRASES (comma-separated) none; an explicit [] disables hints
endpointing (STT) GOOGLE_STT_ENDPOINTING supershort (or short, standard)
voice_name (TTS) GOOGLE_TTS_VOICE fi-FI-Chirp3-HD-Charon

An explicit project="" raises ValueError. TTS supports Chirp 3 HD voices only (streaming synthesis); the language is taken from the voice name.

Callback timing. The providers implement SttProvider and TtsProvider. Chirp 3 sends no interim results. Since 0.6.1 (VR-015), on_recognizing("") fires at speech start and then every 0.25 s while Google reports active speech, so silence or liveness timers driven by recognizing events see a steady cadence, as with Azure. The pulse stops at speech end, a final result, stream end, stop() or a 30 s cap, and is suppressed while speaking. It is not full Azure equivalence: there is no event between Google's speech end and the final transcript (0.4-0.9 s observed), after the cap, or on a replacement stream before its next speech start. Consumers with sub-second silence thresholds should measure their residual cut-off rate.

Commit log line. Google STT committed lang=<code> commit_lag_ms=<int> chars=<int>. commit_lag_ms is the time from the end of speech detected by Google to the final transcript, or -1 when no end-of-speech event preceded it. The transcript text is never logged.

Pacing and stream limits. Audio is sent at most at 1x real time, in 100 ms requests. Google closes a streaming connection after 5 minutes, so the STT replaces it: at the first final transcript after 270 s (ordinary swap), or forcibly at 290 s if the caller is still talking. An ordinary swap can split the utterance in flight across the old and the new stream, so one word may arrive as two pieces in separate final transcripts (live probe: kohta norsu → kohtan + orsu). A forced swap at 290 s may lose the speech in flight, including words spoken before the cut; consumers whose sessions can run past 290 s with continuous speech must accept this. Audio queued while a stream is replaced or reconnected (up to about 7 s of backoff plus a 5 s drain) is sent at the same 1x pace, so it remains as a lasting delay for the rest of the session. Retryable errors reconnect after 1, 2 and 4 s; after three failed replacements, or on an authorisation or argument error, on_error fires once and the session ends.

TTS cancellation. When stop_event is set or the call disconnects, speak() returns within 1 s with interrupted or call_disconnected. If the SDK read cannot be ended within that time, speak() instead fires on_error("google_tts_worker_not_terminated") once and returns {"last_spoken": text, "error": "google_tts_worker_not_terminated"}; the next speak() uses a fresh client.

Live probe. scripts/probe_vr013_google_session.py --project <p> feeds 300 s of synthetic "Kohta <word>." phrases through both providers and checks for an ordinary rotation and for runs of 3 or more consecutive unmatched words, compared case-, punctuation- and whitespace-insensitively; --rescore <log> re-scores an earlier log offline. The 3-word rule detects a pattern, not its cause: lost audio can remove one or two words, and misrecognition can spoil three or more. It does not gate inserted text, duplicates or overall quality. In the recorded run (VR-013 evidence) the last two phrases were not transcribed; the cause is unconfirmed.

Speech check command-line tool (0.6.0, VR-014)

python -m voice_runtime.speech_check checks the speech providers with synthetic speech. It ships in the wheel, so it can run where the consumer runs. There is no console-script entry point.

# TTS then STT for every provider pair (default mode: roundtrip)
python -m voice_runtime.speech_check --provider azure --provider elevenlabs --out tmp/check
# One synthesiser, another recogniser
python -m voice_runtime.speech_check --provider azure --stt-provider elevenlabs
# TTS only, or STT on a file you chose
python -m voice_runtime.speech_check --provider google --mode tts
python -m voice_runtime.speech_check --provider azure --mode stt --audio in.wav --text "..."
Option Meaning
--provider NAME repeatable: azure, elevenlabs, google or all (default all); duplicates are dropped, the first occurrence kept
--mode MODE tts, stt or roundtrip (default)
--text TEXT synthesis text and WER reference
--audio PATH stt mode only: raw .ulaw, or 8 kHz mono WAV as μ-law (tag 7) or 16-bit PCM
--stt-provider NAME roundtrip only: the recogniser for every TTS output (default: the same provider)
--out DIR tts/roundtrip: write <provider>.ulaw and a μ-law <provider>.wav
--timeout SECONDS per step (default 30)
--max-wer RATIO WER threshold (default 0.2); only where a reference applies
--json stdout is JSON lines: a meta object, then one result per row

Example (AC-10 live run, Finnish STT):

provider | stt_provider | step | status | first_audio_ms | total_ms | commits | lag_ms | wer | detail
azure | - | tts | PASS | 628 | 775 | - | - | - | bytes=37200 audio_s=4.650 recognizing=- keys=last_spoken
azure | elevenlabs | stt | PASS | - | 10895 | 1 | 1146 | 0.000 | bytes=- audio_s=- recognizing=5 keys=- transcript='Hyvää huomenta! Haluaisin varata ajan lääkärille ensi viikolla.'

The header on stderr shows the voice_runtime version, its file and git SHA (only inside a voice-runtime checkout), the SDK versions, the Google project and location when Google is selected, and STT_LANGUAGE_CODE. The ElevenLabs STT language defaults to en, so set it (for example fi) to match the text.

Text and scoring. In tts and roundtrip mode, the default text is Hyvää huomenta, haluaisin varata ajan lääkärille ensi viikolla. In stt mode, there is no reference unless --text is given, and then no WER is computed. The transcript is the commits joined with spaces. Both sides are lowercased, punctuation is removed and whitespace is collapsed; ä and ö are kept. WER is the word-level edit distance divided by the number of reference words. A row fails only when WER is greater than --max-wer.

Steps. A TTS step fails when no audio is captured, the result has error, the provider's on_error fires, or an --out write fails. An STT step feeds 20 ms frames, then 3 s of silence, and waits until 2 s pass with no new commit. It fails on no commits, on_error, any exception, or a stop() that raises or blocks. lag_ms runs from the last audio frame to the last commit. In roundtrip mode, if the TTS step fails or is skipped, its STT row is SKIP with dependency: tts <status>.

SKIP is not validation. A row is SKIP only for these causes; everything else is FAIL:

  • the provider module is not in this voice_runtime version;
  • the provider's optional dependency is not installed (azure.cognitiveservices.speech, elevenlabs, google.cloud.speech_v2, google.cloud.texttospeech);
  • missing configuration: AZURE_SPEECH_KEY, ELEVENLABS_API_KEY, or neither GOOGLE_SPEECH_PROJECT nor GOOGLE_CLOUD_PROJECT;
  • Google only: no Application Default Credentials (DefaultCredentialsError).

In roundtrip mode, both rows are SKIP, with no synthesis request, when the recogniser cannot run. SKIP reasons name variables and modules, never values.

Exit codes. 0: at least one step ran and every step that ran passed (SKIPs allowed). 1: any FAIL. 2: a usage error, or every step was SKIP.

Timeouts. Each step runs in its own child process (spawn) with a deadline of --timeout seconds. At the deadline the row is FAIL timeout; the child gets 1 s to cancel cleanly (async work runs stop(), a running speak gets its stop event), then is terminated, then killed, so a step ends within --timeout + 3 s and the next provider still runs. A terminated or killed child does not run stop(), and killing it does not cancel requests a remote service already accepted.

Caller recordings. Use synthetic text or audio you made. Do not point --audio at caller or patient recordings on shared terminals: transcripts are printed to stdout.

SttProvider Protocol

All STT providers implement this structural protocol (defined in providers/__init__.py):

class SttProvider(Protocol):
    on_committed: Callable[[str], None] | None    # final transcript for utterance
    on_recognizing: Callable[[str], None] | None  # interim hypothesis (NC-199)
    on_error: Callable[[str], None] | None        # fatal error after reconnect exhausted (NC-258)

    def set_speaking(self, speaking: bool) -> None: ...
    async def start(self, inbound_queue: asyncio.Queue[bytes | None]) -> None: ...
    async def stop(self) -> None: ...
Callback When it fires Typical consumer action
on_committed Final transcript past echo discard window Route to LLM / FSM
on_recognizing Interim hypothesis (may change) Show live transcription UI
on_error Reconnect attempts exhausted (fatal) Transition FSM to error state

Transport starts/stops the provider; the consumer decides routing (queue, dispatch, ignore).

TtsProvider Protocol

All TTS providers implement this structural protocol (NC-260 Gap A):

class TtsProvider(Protocol):
    on_error: Callable[[str], None] | None  # synthesis failure (NC-260 Gap A)

    def speak(
        self,
        text: str,
        session: VoiceSession,
        stop_event: threading.Event | None = ...,
    ) -> dict[str, Any]: ...

speak() returns a dict with keys: last_spoken (str), and optionally call_disconnected (bool) or interrupted (bool). on_error fires on synthesis failures so the FSM doesn't hang in a speaking state.

VoiceSession

Central coordinator between sync tool threads, async transport, and STT/TTS providers.

Audio I/O

Method Thread safety Purpose
put_inbound(data) Any → async Enqueue caller audio (transport calls this)
get_outbound() async only Dequeue agent audio (transport reads this)
put_outbound_sync(data) Sync → async Enqueue agent audio (TTS provider calls this)
clear_inbound() Any Drain stale audio frames

All sync→async bridging uses asyncio.run_coroutine_threadsafe().

Mark Synchronization

Marks let sync tool code block until the transport confirms audio playback reached a point. This is how you know a TTS utterance finished playing before you start listening.

tts.speak("What is your name?", session)
session.send_mark_and_wait("after-question", timeout=10.0)
# Now safe to start listening — caller heard the full question

session.clear_inbound()
transcript = stt.listen(session, timeout=30)
Method Purpose
send_mark_and_wait(name, timeout) Block sync thread until mark echoed
signal_mark_received(name) Called by transport when mark arrives
get_pending_mark() Async — transport reads next mark to send

Transport Intent (NC-154)

Consumers signal what they want; the transport decides how.

session.request_disconnect()     # transport closes connection, call ends
session.request_clear_buffer()   # transport discards buffered audio (barge-in)

Both are thread-safe. The transport watches _disconnect_requested (asyncio.Event) and _clear_queue (asyncio.Queue) and acts in its own protocol's terms — e.g. Twilio closes the WebSocket, which ends the call; SIP would send BYE.

STT Factory

Attach an STT factory and the transport manages its lifecycle automatically:

from voice_runtime.stt import create_stt

session.stt_factory = lambda: create_stt(provider="elevenlabs")
# Transport calls stt_factory() on stream start, stt.stop() on disconnect

Optional secondary STT for parallel logging/comparison (via SttTee):

session.stt_secondary_factory = lambda: create_stt(provider="azure")
# Transport wraps both in SttTee — primary drives on_committed, secondary logs only

STT Ready Hook (NC-260 Gap E)

Wire callbacks after the transport creates the STT instance but before start():

def wire_callbacks(stt: SttProvider):
    stt.on_committed = handle_transcript
    stt.on_recognizing = handle_interim
    stt.on_error = handle_stt_death

session.on_stt_ready = wire_callbacks

This replaces the old pattern of wiring callbacks before attaching the factory. The transport calls on_stt_ready(stt) after construction, guaranteeing callbacks are set before start() fires.

Lifecycle

Method Purpose
signal_ws_connected(stream_sid) Transport calls when connection established
wait_for_ws_connect(timeout) Consumer blocks until connected; raises CallNotAnsweredError
signal_disconnected() Transport calls on hangup
is_disconnected Property — check if call ended
reset() Clear all state for session reuse (multi-call servers)

Audio Monitoring

Optional two-channel mixer for real-time call monitoring (requires ffplay) and WAV recording (NC-235):

from pathlib import Path
from voice_runtime.audio import AudioMixer

# Monitor only (plays mixed audio through ffplay)
mixer = AudioMixer()

# Monitor + record to WAV file (8kHz mono mulaw)
mixer = AudioMixer(record_path=Path("recordings/call_001.wav"))

mixer.start()
session.set_mixer(mixer)
# session.tap_caller() / session.tap_agent() now feed audio to ffplay + WAV

Exceptions

Exception When
MissingStreamUrlError VOICE_STREAM_URL env var not set
CallNotAnsweredError(timeout) WebSocket didn't connect within timeout
CallHangupError Call hung up during a listen operation

Transport: Twilio

WebSocket Handler

Registers a /voice endpoint on a FastAPI app implementing Twilio Media Streams:

from voice_runtime.transports.twilio_ws import register_voice_websocket

app = FastAPI()
register_voice_websocket(app, session)

Runs 5 async tasks on stream start: send_audio, send_marks, watch_disconnect, send_clears, stt (if factory provided).

Call Initiation

from voice_runtime.transports.twilio_call import (
    initiate_outbound_call,
    build_stream_twiml,  # alias: build_stream_xml
)

# Outbound: dial phone, Twilio connects back to /voice WebSocket
call_sid = initiate_outbound_call("+358401234567")

# Inbound webhook: return TwiML that tells Twilio to stream audio to /voice
xml = build_stream_twiml("wss://example.ngrok.io")

Transport: AudioHook (Genesys Audio Connector) — experimental (0.4.0, VR-011)

Experimental, sandbox-only. Use only against a Genesys sandbox org until the VR-011 E-2 field evidence is recorded. RFC 9421 HTTP message signatures are not verified: the API key is the only credential, and it works as a plain bearer token.

Registers a /audiohook endpoint on which Genesys Cloud is the WebSocket client. The upgrade itself is the call: there is no webhook or TwiML step.

from voice_runtime.transports.audiohook_server import register_audiohook_websocket

app = FastAPI()
register_audiohook_websocket(
    app,
    session,
    api_key=os.environ["AUDIOHOOK_API_KEY"],  # read by the consumer, not the library
    organization_id="<genesys org uuid>",     # optional; checks header and open
    mark_latency_s=0.2,                       # byte-clock mark delay
)
  • Auth: constant-time X-API-KEY comparison, plus the organization ID when set. A failure gets a generic disconnect (unauthorized) after ≥0.5 s. Signature headers are only logged as present or absent.
  • Media: accepts only the offered PCMU 8000 Hz ["external"] entry and echoes it in opened. Caller audio is re-chunked into 160-byte frames for put_inbound. Bot audio is sent as 1600-byte (200 ms) packets, at most 5 per second.
  • Marks are byte-clock emulated. AudioHook has no wire mark. A mark resolves mark_latency_s after the audio queued before it has been sent. This is not a confirmation that the caller heard it. playback_* events are logged as evidence only.
  • Barge-in: request_clear_buffer drops buffered audio, sends an event with a barge_in entity and resolves pending marks.
  • Ending: request_disconnect sends disconnect (completed) with empty outputVariables, then waits up to 10 s for Genesys close. A Genesys close gets closed.
  • Probe connections (nil UUID 00000000-… as conversationId or participant.id) are answered but never bound to the session.
  • Single call per session: one VoiceSession serves one call at a time, as with the Twilio transport.

Providers

ElevenLabs TTS

Streams text → ElevenLabs API → ffmpeg (MP3 → μ-law 8kHz) → session outbound queue.

from voice_runtime.tts import create_tts

tts = create_tts(provider="elevenlabs")
result = tts.speak("Hello", session, stop_event=barge_in_event)
# result: {"last_spoken": "Hello"} or {"last_spoken": "Hello", "call_disconnected": True}

Supports barge-in interrupt: pass a threading.Event as stop_event; set it from another thread to cut TTS mid-stream. Result may include {"interrupted": True}.

ElevenLabs STT

Persistent Scribe WebSocket per call lifetime. Barge-in detection, echo discard, reconnect on fatal errors.

from voice_runtime.stt import create_stt

stt = create_stt(provider="elevenlabs")
stt.on_committed = lambda text: print(f"Heard: {text}")
await stt.start(session.inbound)
# ... later
await stt.stop()

Typically managed by the transport via session.stt_factory rather than started manually.

Azure TTS

Streams text → Azure Speech SDK → mulaw 8kHz → session outbound queue.

from voice_runtime.tts import create_tts

tts = create_tts(provider="azure")
result = tts.speak("Hello", session, stop_event=barge_in_event)

Same speak() interface as ElevenLabs. Uses AZURE_SPEECH_KEY, AZURE_SPEECH_REGION, AZURE_TTS_VOICE env vars.

Azure STT

Persistent Azure Speech SDK recognizer with continuous recognition.

from voice_runtime.stt import create_stt

stt = create_stt(provider="azure", language_code="fi-FI", silence_timeout_ms=1500)
stt.on_committed = lambda text: print(f"Heard: {text}")
await stt.start(session.inbound)

Same SttProvider protocol as ElevenLabs. Echo discard window (400ms) after TTS ends.

Language identification (VR-009, opt-in). Pass candidate locales with languages= or set AZURE_STT_LANGUAGES (comma-separated full locales):

stt = create_stt(provider="azure", languages=["fi-FI", "sv-SE", "en-GB"])
  • Unset or empty: single-locale mode with language_code, unchanged.
  • One locale: single-locale mode with that locale.
  • Two or more: continuous language identification; each phrase is recognized in one of the candidates. Azure always returns one of the candidates, even for speech in another language.
  • Full locales only (fi-FI, not fi), at most 10, and each base language once (not en-US with en-GB). Invalid input raises ValueError; an invalid AZURE_STT_LANGUAGES fails at import of the Azure STT module.
  • An explicit languages= (including []) overrides the env variable.
  • Every forwarded transcript logs Azure STT committed lang=<locale> commit_lag_ms=<int> chars=<int> (detected or configured locale; transcript text is never logged). on_committed still receives only the text.

SttTee (Dual STT)

Fan-out adapter running two STT providers on the same audio stream:

from voice_runtime.stt_tee import SttTee

tee = SttTee(primary=elevenlabs_stt, secondary=azure_stt)
tee.on_committed = handler  # proxied to primary only
await tee.start(session.inbound)

Primary drives production (on_committed). Secondary receives same frames for logging/comparison only — its errors never propagate. Typically wired automatically via session.stt_secondary_factory.

Audio Codec

G.711 μ-law at 8kHz — Twilio's native format. 160 bytes = 20ms frame.

from voice_runtime.audio import mix_frames

mixed = mix_frames(caller_chunk, agent_chunk)  # mix two 160-byte frames

Environment Variables

Variable Purpose Required
VOICE_STREAM_URL Public WebSocket URL for transport callback Yes
VOICE_SERVER_PORT Uvicorn listen port No (default: 8080)
TWILIO_ACCOUNT_SID Twilio credentials (call initiation) For outbound
TWILIO_AUTH_TOKEN Twilio credentials For outbound
TWILIO_PHONE_NUMBER Outbound caller ID For outbound
ELEVENLABS_API_KEY ElevenLabs authentication For ElevenLabs
ELEVENLABS_VOICE_ID TTS voice No (default: Rachel)
ELEVENLABS_MODEL TTS model No (default: eleven_multilingual_v2)
STT_MODEL_ID ElevenLabs STT model No (default: scribe_v2_realtime)
STT_LANGUAGE_CODE ElevenLabs STT language No (default: fi)
AZURE_SPEECH_KEY Azure Speech SDK authentication For Azure
AZURE_SPEECH_REGION Azure Speech SDK region No (default: westeurope)
AZURE_TTS_VOICE Azure TTS voice name No (default: fi-FI-NooraNeural)
AZURE_STT_LANGUAGES Azure STT candidate locales, comma-separated; 2+ enables continuous language identification (VR-009) No (default: unset = language_code)
VOICE_MONITOR Enable AudioMixer monitoring No (default: off)

Transport: SMS (NC-193)

Send SMS via Twilio REST API without importing the Twilio SDK in consumers:

from voice_runtime.transport import get_sms_transport

sms = get_sms_transport()  # default: twilio
result = sms.send_sms(to="+358401234567", body="Your appointment is confirmed.")
# result: {"message_sid": "SM...", "status": "queued", "to": "+358..."}

Requires TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, and TWILIO_PHONE_NUMBER env vars.

Consumer Pattern

Typical consumer subclasses VoiceSession and adds server lifecycle:

from dataclasses import dataclass
from voice_runtime.session import VoiceSession

@dataclass
class TelcoSession(VoiceSession):
    def start(self):
        app = FastAPI()
        register_voice_websocket(app, self)
        # Run uvicorn in daemon thread
        threading.Thread(target=self._run_loop, daemon=True).start()

    def shutdown(self):
        # Signal event loop to stop, join thread
        ...

Tool nodes then use the session for audio I/O, mark sync, and transport intents — without knowing anything about Twilio, WebSockets, or ElevenLabs.

Known Consumers

Any project that subclasses VoiceSession and registers a FastAPI WebSocket handler via register_voice_websocket is a consumer. See the Consumer Pattern section above for the canonical implementation.

Multi-Call Session Reuse

When servers handle multiple sequential calls on the same VoiceSession instance, reset() clears all state between calls:

  • Stops active STT via asyncio.run_coroutine_threadsafe(stt.stop(), loop) before clearing the reference (prevents orphaned WebSocket connections)
  • Drains inbound and outbound queues
  • Resets mark synchronization and transport intent events

The STT start() method also drains the inbound queue as defense-in-depth against sentinel values from prior call cleanup.

STT Reconnect on Fatal Errors

PersistentSttSession detects fatal WebSocket errors (connection closed, protocol errors) and automatically reconnects:

  • _on_error() schedules _reconnect_after_error() for errors in _FATAL_ERRORS
  • Reconnect drains stale frames, creates a new WebSocket, and resumes the feed task
  • _feed_audio() wraps send() in try/except for dead socket resilience

Metadata

Release files for voice-runtime 0.6.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for voice-runtime 0.6.1
File Size Uploaded Provenance Actions
voice_runtime-0.6.1.tar.gz 166.9 kB With provenance Details for voice_runtime-0.6.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for voice-runtime 0.6.1
File Interpreter ABI Platform Actions
voice_runtime-0.6.1-py3-none-any.whl Python 3 none any Details for voice_runtime-0.6.1-py3-none-any.whl

Total release size: 250.9 kB

Release files / voice_runtime-0.6.1.tar.gz

Download URL voice_runtime-0.6.1.tar.gz
Size 166.9 kB
Tags Source
SHA-256 checksum
How to use checksums
295cf751c8886ddd1fd60d6a006c4c71cd48e87ab12015e9ec105f18853ba949
BLAKE2b-256 checksum
How to use checksums
4da8eb888a80f4b322aa78d39b3a13bd13b014e8de78283b506f1af824a6f753
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 17, 2026.

Transparency log

Release files / voice_runtime-0.6.1-py3-none-any.whl

Download URL voice_runtime-0.6.1-py3-none-any.whl
Size 84.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
332b30898bb30b636773c090abe92ff1f0065271a893752015fe338b0e753a6f
BLAKE2b-256 checksum
How to use checksums
9dab0c090005e3ecd428c152a73e71cc501a90021c21781e20d7b5daabba2792
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 17, 2026.

Transparency log

Security for voice-runtime 0.6.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for voice_runtime-0.6.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 17, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page