livekit-plugins-tts-cache-router

LiveKit Agents plugin for tts-cache-router: Sarvam, Raya, ElevenLabs, OpenAI, Deepgram, xAI, Soniox, Smallest AI and Cartesia text-to-speech, with every sentence your agent has said before served from a per-account cache.

Same output whether you call the provider directly or through the router: each sentence reaches the provider verbatim with your exact settings, and a cache hit replays a real provider take for that exact request.

Full documentation: docs.ttscacherouter.dev/livekit. For LLMs and coding agents: llms.txt, and this page as Markdown at livekit.md.

Install

pip install livekit-plugins-tts-cache-router

Usage

from livekit.plugins import tts_cache_router as tcr

tts = tcr.TTS(model="sarvam/bulbul:v3", voice="shubh", language="hi-IN")
# or
tts = tcr.TTS(model="raya/standard", voice="d6a002d0-230c-49b1-a137-b8a7d564b1ae", language="hi")
# or
tts = tcr.TTS(
    model="elevenlabs/eleven_flash_v2_5",
    voice="JBFqnCBsd6RMkjVDRZzb",
    options={"voice_settings": {"stability": 0.5, "similarity_boost": 0.75}},
)
# or
tts = tcr.TTS(model="openai/gpt-4o-mini-tts", voice="ash", options={"instructions": "Speak warmly."})
# or
tts = tcr.TTS(model="deepgram/aura-2", voice="thalia-en")  # Deepgram's aura-2-thalia-en
# or
tts = tcr.TTS(model="xai/xai-tts", voice="ara", language="auto")
# or
tts = tcr.TTS(model="soniox/tts-rt-v2", voice="Adrian", language="en")
# or
tts = tcr.TTS(model="smallestai/lightning_v3.1_pro", voice="meher", language="en")
# or
tts = tcr.TTS(model="cartesia/sonic-3", voice="f786b574-daa5-4673-aa0c-cbe3e8534c02", language="en")

session = AgentSession(tts=tts, ...)

Before the first request, store your Sarvam, Raya, ElevenLabs, OpenAI, Deepgram, xAI, Soniox, Smallest AI or Cartesia API key in the dashboard at ttscacherouter.dev. The router keeps it encrypted and uses it only to call that provider for your requests.

Set TTS_CACHE_ROUTER_API_KEY to your router API key, from the same dashboard (or pass api_key=).

TTS(...)

The settings are in the router's own names (those of /v1/audio/speech and GET /v1/models), the same for every provider:

Every provider's models, voices, languages, sample rates and options are in Models and options. The router checks the settings: when the session starts, the plugin connects and a wrong setting is logged as an error, and a reply with a wrong one fails with APIStatusError (bad_request, not retryable).

How long cached audio is kept is your org's retention setting, in the dashboard.

Changing settings mid-session

tts.update_options(voice="priya", speed=1.1)
tts.update_options(model="cartesia/sonic-3", voice="f786b574-daa5-4673-aa0c-cbe3e8534c02", language="en")

Replies that start afterwards use the new settings. options replaces the options as a whole. A model of another provider starts from no language, speed or options unless they are given too.

Provider notes

• ElevenLabs: eleven_v3 and eleven_v4 models go through ElevenLabs' text-to-dialogue API. Transcripts are word by word (turn this off with options={"sync_alignment": False}).
• OpenAI speaks the language of the text, at 24000 Hz only, and takes no language.
• Deepgram: the model is the family and the voice the rest of Deepgram's model id (model="deepgram/aura-2", voice="thalia-en" is aura-2-thalia-en). With Aura, the first sentence of a reply after a pause of about five seconds or more takes about 0.7–1.2 s longer to start than with LiveKit's Deepgram plugin; sentences from the cache, and Flux, are unaffected.
• Soniox needs language: it reads the text through that language's phonetics.
• xAI, Soniox, Smallest AI and Cartesia: each sentence is synthesized on its own, so the pauses between sentences are a little shorter than in one provider stream, and a sentence doesn't carry the prosody of the one before it.
• Cartesia: transcripts are word by word (turn this off with options={"word_timestamps": False}).

Connections

When an AgentSession starts, it calls tts.prewarm(): the plugin connects to the router, and the router connects to your provider with the current settings at once, so the first reply doesn't wait for either. Idle connections stay open between replies, for up to about 30 minutes, and are replaced at once when the router restarts.

Interruptions

When the agent is interrupted, the plugin cancels the router segment and keeps the connection open, so the next reply starts on a warm connection.

Metrics

TTSMetrics.characters_count counts only characters the provider synthesized (and billed) for the request; sentences served from the cache count zero. provider and model report the underlying provider and model, so usage dashboards attribute cost correctly.

Once a reply is all sent, the plugin also emits cache_stats, a CacheStats with the reply's request_id, segment_id (when streamed), sentences, cached_sentences, characters and billed_characters (and cached_characters, the difference):

tts.on("cache_stats", lambda stats: print(stats.cached_sentences, "of", stats.sentences, "cached"))