pipecat-slng

Built and maintained by the SLNG team (slng.ai).

WebSocket STT and TTS services for Pipecat, backed by SLNG — a unified voice AI gateway that routes to multiple STT/TTS providers (Deepgram, ElevenLabs, Rime, Sarvam, and more) through a single API key. Swap the model string to switch providers; no other code changes needed.

Requires Pipecat v1.8.0 or newer, and is tested against v1.8.0. Earlier pipecat-slng releases do not import on Pipecat 1.8.0.

Installation

uv add pipecat-slng
# or
pip install pipecat-slng

Environment variables

SLNG_API_KEY=your_slng_api_key      # get one at https://slng.ai
OPENAI_API_KEY=your_openai_api_key  # only needed for the example bot (LLM)

Copy .env.example to .env to get started.

Usage (streaming WebSocket — recommended)

SlngSTTService and SlngTTSService run over WebSocket: low-latency, supports mid-utterance interruption, and exposes the full SLNG config surface (encoding, sample_rate, language, speed).

import os

from pipecat_slng import SlngSTTService, SlngTTSService

stt = SlngSTTService(
    api_key=os.getenv("SLNG_API_KEY"),
    model="slng/deepgram/nova:3-en",
)

tts = SlngTTSService(
    api_key=os.getenv("SLNG_API_KEY"),
    model="slng/deepgram/aura:2-en",
    voice="aura-2-thalia-en",
)

Common runtime knobs are top-level kwargs (e.g. language=, speed=, enable_vad=, enable_partials=). For richer overrides pass a SlngSTTSettings(...) / SlngTTSSettings(...) to settings=.

Defaults when not specified: STT uses language=Language.EN, enable_vad=True, enable_partials=True; TTS uses language=Language.EN and the server's default speed.

Three behaviors worth knowing:

• Confidence filter (STT). When the provider surfaces a confidence score, partial transcripts below 0.5 are dropped. Finals are never dropped — discarding one hangs the user turn rather than losing a word.

• Turn finalization (STT). On VADUserStoppedSpeakingFrame the service sends finalize and marks the answering transcript TranscriptionFrame.finalized, which is what lets Pipecat end the user turn immediately instead of waiting out its safety-net timer. That timer is sized by ttfs_p99_latency, for which this service declares no default, so Pipecat substitutes a conservative 1.0 s and logs a warning at pipeline start. It only affects the fallback path — a finalized transcript cancels the timer before it fires — so the warning is cosmetic.

  To right-size the fallback, pass your own value: SlngSTTService(..., ttfs_p99_latency=<seconds>). Measure it with stt-benchmark at VADParams.stop_secs=0.2, the threshold Pipecat's built-in values assume. Indicative figures from our own harness (speech-end → final transcript, small sample, one network path — not a substitute for a benchmarked P99): ~605 ms median on slng/deepgram/nova:3-en and ~280 ms on deepgram/nova:3. Prefer a value above your observed maximum: too low expires the safety net early and cuts the caller off, which is worse than waiting.

• Runtime settings updates. Changing voice, speed, or language mid-session (via Pipecat settings updates) reconnects the WebSocket to re-run the init handshake — expect a brief reconnect, not a silent no-op.

Optional TTS warm standby

Enable one prepared connection for the next utterance:

tts = SlngTTSService(
    api_key=os.environ["SLNG_API_KEY"],
    model="gradium/tts:default",
    voice="QETTJoT4n_WmpL3w",
    warm_standby_enabled=True,  # default: False
)

Preparation starts after the first successful text send. The next utterance uses the spare only after that connection acknowledges ready; a pending, expired, or failed spare falls back to ordinary connection handling. Startup still uses the initial connection. All fragments of an utterance stay together; Pipecat's default reuse_context_id_within_turn=True is required.

When enabled, the service rotates to a ready spare even if the old connection remains open. It waits for preceding synthesis to finish, preserves queued playback, and closes the old transport in the background. At most two connections are opening, open, or closing. Settings changes invalidate prepared connections; interruption retires unfinished synthesis; shutdown closes both. Preparation uses an extra gateway/provider session, which can consume connection quota or incur provider charges even when unused.

Both initialized connections receive the existing 30-second keepalive. This cannot prevent every provider expiry, and gateway ready does not guarantee that all provider preparation has finished. An expired spare is retried after later speech, without synthetic warmup text or a continuous reconnect loop. Debug logs report actual use, miss reason, and background preparation time by synthesis context, without logging speech or credentials in those records.

In September 2026 tests with Pipecat 1.8.0, Gradium's controlled per-utterance reconnection case improved median request-to-first-audio from 1,079–1,415 ms across baseline batches to 77–109 ms with standby. Each condition used five utterances, interleaving baseline/standby/baseline twice with 1.5-second gaps; these medians exclude startup. Reconnection was forced by the test to isolate its cost. Normal Gradium, Deepgram, and Sarvam reuse showed no consistent improvement. These client measurements do not identify deployed gateway revision or provider cache state; measure your route before enabling the option.

The opt-in measurement test supports SLNG_TTS_STANDBY_COMPARE=1, alongside SLNG_API_KEY, SLNG_TTS_MEASURE=1, and explicit SLNG_TTS_MODEL and SLNG_TTS_VOICE. SLNG_TTS_FORCE_RECONNECT=1 selects the labelled controlled condition. These are test inputs, not service configuration.

Pronunciation dictionaries

Streaming TTS can use one SLNG pronunciation dictionary as the WebSocket session default. Reference it by name or immutable ID; SLNG validates and applies the rewrite rules.

tts = SlngTTSService(
    api_key=os.getenv("SLNG_API_KEY"),
    model="slng/deepgram/aura:2-en",
    voice="aura-2-thalia-en",
    pronunciation={"mode": "rewrite", "name": "support-pronunciations"},
)

Use {"mode": "rewrite", "dictionary_id": "pd_..."} to reference an immutable dictionary version. See the pronunciation dictionary docs.

HTTP TTS (non-streaming fallback)

For simple request/response synthesis where streaming is not required, use SlngHttpTTSService. It issues one HTTP POST per utterance and returns the full audio body in one frame.

import os

from pipecat_slng import SlngHttpTTSService

tts = SlngHttpTTSService(
    api_key=os.getenv("SLNG_API_KEY"),
    model="slng/deepgram/aura:2-en",
    voice="aura-2-thalia-en",
)

HTTP contract limits. Per the SLNG Unified TTS HTTP OpenAPI, the request body accepts only {text, voice} — there is no config object. Encoding, sample_rate, language, and speed are therefore not configurable over HTTP; the server returns its default audio format. The service auto-detects WAV (decoded to raw PCM at the file's sample rate) and plain PCM (passed through at the pipeline's sample rate). Compressed responses (MP3/Ogg) yield an ErrorFrame — use the streaming SlngTTSService if you need codec control.

An aiohttp.ClientSession is created internally if you don't pass one; supply aiohttp_session=... to reuse a shared session.

Region routing

Both services support gateway region routing via region_override (pin to a datacenter: ap-southeast-2 | eu-north-1 | us-east-1) and world_part_override (broad zone: ap | eu | na). When both are set, region_override wins. WebSocket services send these as the X-Region-Override / X-World-Part-Override headers; the HTTP service uses the region / world-part query parameters (per the bridge contract).

stt = SlngSTTService(
    api_key=os.getenv("SLNG_API_KEY"),
    model="slng/deepgram/nova:3-en",
    region_override="eu-north-1",
)

Model routing & bring-your-own-key (BYOK)

The model string decides where transcription/synthesis runs:

• slng/... (e.g. slng/deepgram/aura:2-en) — hosted by SLNG.
• anything else (e.g. deepgram/aura:2, elevenlabs/..., cartesia/sonic:3, sarvam/bulbul:v3) — an external provider, proxied through SLNG.

An external route works with or without your own provider key — those are two independent choices. The slng/ prefix is what selects SLNG-hosted; BYOK is a separate decision layered on top. The full matrix:

For BYOK, pass your own provider key via provider_key. It is forwarded as the X-Slng-Provider-Key header, so the provider bills your account directly and no SLNG audio-minute fees apply — the SLNG cache still applies on top. This is a separate key from SLNG_API_KEY, which always authenticates you to SLNG. See the BYOK docs.

# BYOK = an external route + your own provider key. Deepgram is shown here; the
# same pattern works for any external provider (ElevenLabs, Cartesia, Sarvam, …).
stt = SlngSTTService(
    api_key=os.getenv("SLNG_API_KEY"),            # authenticates you to SLNG
    model="deepgram/nova:3",                      # external route — no slng/ prefix
    provider_key=os.getenv("SLNG_PROVIDER_KEY"),  # your own provider key
)

tts = SlngTTSService(
    api_key=os.getenv("SLNG_API_KEY"),
    model="deepgram/aura:2",                      # external route — no slng/ prefix
    voice="aura-2-thalia-en",
    provider_key=os.getenv("SLNG_PROVIDER_KEY"),
)

BYOK is valid only on external routes; an slng/... route plus a provider_key is rejected with a 400 ("BYOK is only supported for external STT/TTS routes"). If the provider rejects your key, the failure surfaces as a backend_connection_failed error frame over WebSocket, or the upstream 401/403 with the X-Slng-Auth-Source: client_key response header over HTTP.

Example

A complete cascade bot (STT → LLM → TTS, WebSocket TTS by default) lives in examples/bot.py:

cp .env.example .env   # fill in SLNG_API_KEY and OPENAI_API_KEY
uv run --extra example examples/bot.py

Then open http://localhost:7860/client in your browser and start talking. Pick models with SLNG_STT_MODEL / SLNG_TTS_MODEL (both default to slng/... self-hosted routes); set SLNG_PROVIDER_KEY to your own provider key to run an external route in BYOK mode. The bot uses the SmallWebRTC transport by default; pass -t daily to use Daily instead (requires installing pipecat-ai[daily]).

Development

uv sync --all-extras
uv run pytest          # unit tests (live smoke tests skip without SLNG_API_KEY)
uv run ruff check .
uv run ty check .

About SLNG

SLNG (https://slng.ai) is a unified voice AI gateway. Learn more in the SLNG docs.

License

BSD-2-Clause — see LICENSE.