Baseten plugin for LiveKit Agents

Support for Baseten-hosted models in LiveKit Agents, including STT (Speech-to-Text), TTS (Text-to-Speech), and LLM (Large Language Model) integrations.

Installation

pip install livekit-plugins-baseten

Pre-requisites

You'll need an API key from Baseten. It can be set as an environment variable: BASETEN_API_KEY

You also need to deploy a model to Baseten and will need your model endpoint to configure the plugin.

STT (Speech-to-Text)

The STT plugin connects to Baseten's Whisper Streaming WebSocket endpoint for real-time transcription. It works with both truss and chain deployments.

Recommended model

Whisper v3 Turbo – WebSocket

Endpoint URL formats

Deployment type URL pattern
Truss wss://model-{model_id}.api.baseten.co/environments/production/websocket
Chain wss://chain-{chain_id}.api.baseten.co/environments/production/websocket

Basic usage

You can specify the endpoint in three ways:

from livekit.plugins import baseten

# 1. Using a truss model ID (recommended for truss deployments)
stt = baseten.STT(
    api_key="your-baseten-api-key",  # or set BASETEN_API_KEY env var
    model_id="your-model-id",
    language="en",
)

# 2. Using a chain ID (recommended for chain deployments)
stt = baseten.STT(
    api_key="your-baseten-api-key",
    chain_id="your-chain-id",
    language="en",
)

# 3. Using a full endpoint URL (for custom routing or deployment URLs)
stt = baseten.STT(
    api_key="your-baseten-api-key",
    model_endpoint="wss://model-{model_id}.api.baseten.co/environments/production/websocket",
    language="en",
)

Configuration options

Parameter Default Description
api_key BASETEN_API_KEY env var Baseten API key
model_endpoint BASETEN_MODEL_ENDPOINT env var Full WebSocket URL (takes priority over model_id/chain_id)
model_id — Baseten truss model ID; auto-constructs the endpoint URL
chain_id — Baseten chain ID; auto-constructs the endpoint URL
language "en" BCP-47 language code (use "auto" for auto-detection)
language_options [] Restrict auto-detection to these codes, e.g. ["en", "de"]. More reliable than "auto" on short telephony utterances. Requires Whisper runtime v0.5.0+
encoding "pcm_s16le" Audio encoding (pcm_s16le or pcm_mulaw)
sample_rate 16000 Audio sample rate in Hz
enable_partial_transcripts True Emit interim transcripts while the speaker is talking
partial_transcript_interval_s 1.0 Interval (seconds) between partial transcript updates
final_transcript_max_duration_s 30 Max seconds of audio before forcing a final transcript
show_word_timestamps True Include word-level timestamps in results
vad_threshold 0.5 Server-side VAD speech probability threshold (0.0–1.0)
vad_min_silence_duration_ms 300 Minimum silence (ms) to mark end of speech
vad_speech_pad_ms 30 Padding (ms) added around detected speech

Full voice pipeline example

import os
from livekit import agents
from livekit.agents import AgentSession, Agent, RoomInputOptions, inference
from livekit.plugins import baseten, openai, noise_cancellation
from livekit.agents.inference import TurnDetector

BASETEN_API_KEY = os.getenv("BASETEN_API_KEY")
whisper_model_id = "your-whisper-model-id"  # or use chain_id for chain deployments
orpheus_model_id = "your-orpheus-model-id"


class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(instructions="You are a helpful voice AI assistant.")


async def entrypoint(ctx: agents.JobContext):
    session = AgentSession(
        stt=baseten.STT(
            api_key=BASETEN_API_KEY,
            model_id=whisper_model_id,  # or chain_id="your-chain-id"
            language="en",
            enable_partial_transcripts=True,
        ),
        llm=openai.LLM(
            api_key=BASETEN_API_KEY,
            base_url="https://inference.baseten.co/v1",
            model="openai/gpt-oss-120b",
        ),
        tts=baseten.TTS(
            api_key=BASETEN_API_KEY,
            model_endpoint=(
                f"https://model-{orpheus_model_id}"
                ".api.baseten.co/environments/production/predict"
            ),
        ),
        vad=inference.VAD(),
        turn_detection=TurnDetector(),
    )

    await session.start(
        room=ctx.room,
        agent=Assistant(),
        room_input_options=RoomInputOptions(
            noise_cancellation=noise_cancellation.BVC(),
        ),
    )

    await session.generate_reply(
        instructions="Greet the user and offer your assistance."
    )


if __name__ == "__main__":
    agents.cli.run_app(agents.WorkerOptions(entrypoint_fnc=entrypoint))

TTS (Text-to-Speech)

The TTS plugin calls Baseten-hosted TTS models (e.g. Orpheus 3B) over HTTP.

tts = baseten.TTS(
    api_key="your-baseten-api-key",
    model_endpoint="https://model-{model_id}.api.baseten.co/environments/production/predict",
    voice="tara",
    language="en",
)

Qwen3 (STT + TTS)

Baseten also hosts Qwen3-ASR and Qwen3-TTS. They speak different wire protocols from the Whisper/Orpheus trusses, so select them with model — the same baseten.STT / baseten.TTS classes handle both.

from livekit.agents import AgentSession
from livekit.plugins import baseten

session = AgentSession(
    stt=baseten.STT(model="qwen3-asr", model_id="your-qwen3-asr-model-id"),
    tts=baseten.TTS(model="qwen3-tts", model_id="your-qwen3-tts-model-id",
                    voice="your-voice"),
    # llm=...
)
model="whisper" / "orpheus" (default) model="qwen3-asr" / "qwen3-tts"
STT audio raw binary PCM base64 input_audio_buffer.append
STT results message_type / transcript type: "transcription" / segments[].text
TTS transport HTTP, or WS with an __END__ sentinel session.config → input.text → input.done
TTS voices preset names (tara) registered voice clones

Several defaults follow the selected model, matching each deployment's own:

Parameter whisper / orpheus qwen3-asr / qwen3-tts
language en auto (STT) / Auto (TTS)
partial_transcript_interval_s 1.0 0.5
vad_min_silence_duration_ms 300 500
vad_speech_pad_ms 30 100
show_word_timestamps on off (needs STREAM_ALIGNER=mms on the deployment)
voice tara required — a registered clone

Qwen3 TTS voices

Qwen3-TTS Base ships no built-in speakers — there is no tara equivalent. Register a clone from 10–20s of clean speech, then pass its name as voice:

from livekit.plugins.baseten import list_voices, register_voice

await register_voice(
    model_endpoint="wss://model-{model_id}.api.baseten.co/environments/production/websocket",
    name="my_voice",
    ref_audio_path="./reference.wav",
    ref_text="Transcript of the reference audio.",
)
await list_voices(model_endpoint=...)  # {"voices": [...], "uploaded_voices": [...]}

The server stores uploaded voices on the container's local disk, so a voice registered at runtime lives on one replica and is lost when that container restarts. For anything beyond single-replica testing, bake the reference audio into the deployment (REQUIRED_VOICES) so every replica starts with it, or pass ref_audio/ref_text to clone inline on each session.

Qwen3-only options

task_type (Base cloning, or CustomVoice/VoiceDesign deployments), instructions, max_new_tokens, initial_codec_chunk_frames, x_vector_only_mode, ref_audio/ref_text, and extra_config (merged into session.config for server fields newer than this plugin) apply to model="qwen3-tts" only.

LLM (Large Language Model)

The LLM plugin wraps Baseten's OpenAI-compatible inference endpoint.

llm = baseten.LLM(
    api_key="your-baseten-api-key",
    model="openai/gpt-oss-120b",
)

Documentation

Project links

Data verified by PyPI on Aug 13, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.7.0)

Owner

PyPI data
Data sourced directly from PyPI's database.

Credits

Author: LiveKit

GitHub Statistics

Data verified by PyPI on Aug 13, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python >=3.10.0

Tags

ai audio livekit realtime video voice webrtc

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

livekit_plugins_baseten-1.6.10.tar.gz (26.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

livekit_plugins_baseten-1.6.10-py3-none-any.whl (34.5 kB view details)

Uploaded Python 3

File details

Details for the file livekit_plugins_baseten-1.6.10.tar.gz.

File metadata

  • Download URL: livekit_plugins_baseten-1.6.10.tar.gz
  • Upload date:
  • Size: 26.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for livekit_plugins_baseten-1.6.10.tar.gz
Algorithm Hash digest
SHA256 c17a664e69caeb21a365462cdac5ba8b674e17901364efab2535d7a6943d7b0f
MD5 78c8393cccb8cfdfb70234bd7f6e6eec
BLAKE2b-256 3eb22afa02b5a461b8d165f83cd0d84d33d282896a3f11dad9d365d2af6c2354

See more details on using hashes here.

Provenance

The following attestation bundles were made for livekit_plugins_baseten-1.6.10.tar.gz:

Publisher: publish.yml on livekit/agents

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file livekit_plugins_baseten-1.6.10-py3-none-any.whl.

File metadata

File hashes

Hashes for livekit_plugins_baseten-1.6.10-py3-none-any.whl
Algorithm Hash digest
SHA256 d5cb750b2b884a6f937cbe6e2869357dd591648a11c39377e60284d45bdb0482
MD5 4d5aa7a1a373800ba03b845b76a04036
BLAKE2b-256 3007a4d917de2df99c37114037410e9d9e13f130aea91ac8d97c2e4be30b28c1

See more details on using hashes here.

Provenance

The following attestation bundles were made for livekit_plugins_baseten-1.6.10-py3-none-any.whl:

Publisher: publish.yml on livekit/agents

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

1.7.0

2 files

Pre-release

1.6.11rc1

2 files

This release

1.6.10 This release

2 files

1.6.9

2 files

1.6.8

2 files

1.6.7

2 files

1.6.6

2 files

1.6.5

2 files

1.6.4

2 files

1.6.3

2 files

1.6.2

2 files

1.6.1

2 files

1.6.0

2 files

Pre-release

1.6.0rc2

2 files

Pre-release

1.6.0rc1

2 files

Pre-release

1.5.19rc1

2 files

1.5.18

2 files

1.5.17

2 files

1.5.16

2 files

1.5.15

2 files

1.5.14

2 files

1.5.13

2 files

1.5.12

2 files

1.5.11

2 files

1.5.10

2 files

1.5.9

2 files

1.5.8

2 files

1.5.7

2 files

1.5.6

2 files

1.5.5

2 files

1.5.4

2 files

1.5.3

2 files

1.5.2

2 files

1.5.1

2 files

1.5.0

2 files

Pre-release

1.5.0rc2

2 files

Pre-release

1.5.0rc1

2 files

1.4.6

2 files

1.4.5

2 files

1.4.4

2 files

1.4.3

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

Pre-release

1.4.0rc2

2 files

Pre-release

1.4.0rc1

2 files

1.3.12

2 files

1.3.11

2 files

1.3.10

2 files

1.3.9

2 files

1.3.8

2 files

1.3.7

2 files

1.3.6

2 files

1.3.5

2 files

1.3.4

2 files

1.3.3

2 files

1.3.2

2 files

1.3.1

2 files

Pre-release

1.3.0rc2

2 files

Pre-release

1.3.0rc1

2 files

1.2.18

2 files

1.2.17

2 files

1.2.16

2 files

1.2.15

2 files

1.2.14

2 files

1.2.13

2 files

1.2.12

2 files

1.2.11

2 files

1.2.9

2 files

1.2.8

2 files

1.2.7

2 files

1.2.6

2 files

1.2.5

2 files

1.2.4

2 files

1.2.3

2 files

1.2.2

2 files

1.2.1

2 files

1.2.0

2 files

1.1.7

2 files

1.1.6

2 files

1.1.5

2 files

1.1.4

2 files

1.1.3

2 files

1.1.2

2 files

1.1.1

2 files

1.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page