echospeaks

echospeaks.ai

Sentence-level TTS audio cache for pipecat voice pipelines. Audio lives in echo-server (the pod-shared cache service; it keeps its own L1/L2 internally) - there is no client-side store. Caching is enabled by passing an api_key in cache_config; without it everything passes through untouched and the pipeline behaves fully stock. To get a key, visit echospeaks.ai and contact us.

Layout

echospeaks/
  defaults.py   SDK-wide defaults
  common/       shared dataclasses + logger
  cache/        cache keys + the echo-server client
  pipecat_tts/  the pipecat integration (read.py, write.py, plan.py)

How it works

Two pipeline processors around the host's completely stock TTS service:

• Read - EchoTTSCacheRead (echospeaks.pipecat_tts.read): placed before the tts. Aggregates LLM tokens into sentences (the TTS receives pre-aggregated sentences, so its own aggregator never engages) and asks echo-server per sentence (0.5s bounded) - a hit is swallowed (the TTS never sees it); a miss flows through and is registered in the shared turn plan for capture.
• Write - EchoTTSCacheWrite (echospeaks.pipecat_tts.write): placed right after the tts. Slices each miss sentence's audio out of the turn's shared provider context by word timestamps and fire-and-forgets a POST to echo-server (audio base64-encoded inside the v1 JSON envelope; decoded on fetch), and injects cached hit audio at the exact sentence boundary so sentences always play in LLM order. Interruptions discard partial buffers, so truncated audio is never stored.

Standalone utterances (TTSSpeakFrame) are cached too. Fixed prompts the host pushes directly (never through the LLM) are cached as one entry over the full utterance. A miss forwards the original frame and captures its audio; a hit swallows it and replays the cached audio with the same frame sequence a live utterance produces. Fixed prompts repeat on every call, making them the highest-value entries in the cache.

echo-server behavior is strictly fail-open: the lookup is bounded by a 0.5s timeout and every error (down, slow, 401, 5xx) degrades to a plain miss.

Install

pip install echospeaks

pipecat-ai comes from your own app (the pipeline the processors go into); pip install "echospeaks[pipecat]" pulls it in for a standalone install.

Usage

The TTS service stays completely stock: the cache is two pipeline processors around it. The read side aggregates LLM tokens into sentences (so the TTS receives pre-aggregated sentences and its own aggregator never engages), swallows cache hits, and forwards misses; the paired writer captures each miss sentence's audio for storing and injects cached hit audio at the exact sentence boundary, so sentences always play in LLM order for any hit/miss pattern.

from pipecat.services.elevenlabs.tts import ElevenLabsTTSService
from echospeaks.pipecat_tts import EchoTTSCacheRead, EchoTTSCacheWrite

tts = ElevenLabsTTSService(                          # untouched, fully stock
    api_key=ELEVENLABS_API_KEY,
    settings=ElevenLabsTTSService.Settings(model="eleven_turbo_v2_5", voice=VOICE_ID),
)

EchoTTSCacheRead.configure({
    "api_key": ECHO_SERVER_API_KEY,
    "provider": "elevenlabs",
    "voice_id": VOICE_ID,
    "model_id": "eleven_turbo_v2_5",
    "tag": "my-campaign",
})

pipeline = Pipeline([
    ...,
    llm,
    EchoTTSCacheRead(),    # picks up the configure()d config
    tts,
    EchoTTSCacheWrite(),   # pairs itself with the read upstream at StartFrame
    transport.output(),
    ...,
])

EchoTTSCacheRead.current() returns this call's instance, whose stats property exposes the raw counters for the host's own logs/metrics:

s = EchoTTSCacheRead.current().stats
# {"hits": 6, "misses": 3, "hit_chars": 214, "miss_chars": 101,
#  "total_chars": 315, "total_sentences": 9, "hit_rate": 0.679}
log.info(f"cache hits={s['hits']} misses={s['misses']} hit_rate={s['hit_rate']:.0%}")

hit_rate is character-weighted (characters are what synthesis costs), 0..1.

Without an api_key (or the keying inputs) everything passes through untouched. Segmentation and in-order injection use word timestamps; ElevenLabs and Cartesia both provide them. Without word timestamps the writer degrades gracefully: single-sentence turns still store, mid-turn hits inject at turn end.

TTSSpeakFrames need no extra wiring - they pass through the same read processor and are cached whole. One trade-off to know: a speak utterance waits for its lookup before synthesis starts, so its time-to-first-audio shifts by the lookup round trip (~60-250ms, bounded at 0.5s fail-open) on a miss - and skips synthesis entirely on a hit.

cache_config reference

One dict holds every host tunable. Missing or None keys fall back to the defaults in echospeaks/defaults.py; unknown keys are warned about and ignored (never a crash).

How the dict is sourced is the host's business - env vars, a config file, a per-tenant document; the SDK only sees the final dict.

Tests

uv sync && uv run pytest

Includes an HTTP round-trip test against an in-process fake echo-server (auth, miss, store, hit, delete, fail-open).