Azure Plugin for LiveKit Agents
Complete Azure AI integration for LiveKit Agents, including Azure Speech Services and Azure Voice Live Realtime API.
What's included:
- RealtimeModel - Azure Voice Live for speech-to-speech
- STT - Powered by Azure Speech Services
- TTS - Powered by Azure Speech Services with neural voices
For Azure OpenAI LLM (non-realtime), see the OpenAI plugin which supports LLM.with_azure().
See https://docs.livekit.io/agents/integrations/azure/ for more information.
Installation
pip install livekit-plugins-azure
# For the Azure Voice Live realtime model
pip install "livekit-plugins-azure[realtime]"
Realtime Mode (Azure Voice Live)
Azure Credentials
For the realtime speech-to-speech model:
export AZURE_VOICE_LIVE_ENDPOINT=https://<region>.api.cognitive.microsoft.com/
export AZURE_VOICE_LIVE_API_KEY=<your-speech-key>
export AZURE_VOICE_LIVE_MODEL=<model-name> # e.g., gpt-realtime, gpt-4o, gpt-4o-mini, etc.
To authenticate with Microsoft Entra ID instead of an API key, pass use_default_credential=True, which uses DefaultAzureCredential.
To power the intelligence of your voice agent, you have flexibility and choice in the generative AI model between GPT-Realtime, GPT-5, GPT-4.1, Phi, and more options. For supported models and regions, see Supported models and regions.
Quick Start
Azure Voice Live provides end-to-end speech-to-speech:
from livekit import agents
from livekit.agents import Agent, AgentServer, AgentSession
from livekit.plugins import azure
class Assistant(Agent):
def __init__(self):
super().__init__(
instructions="You are a helpful voice assistant powered by Azure."
)
async def on_enter(self):
await self.session.generate_reply(
instructions="Greet the user and offer assistance"
)
server = AgentServer()
@server.rtc_session()
async def entrypoint(ctx: agents.JobContext):
session = AgentSession(
llm=azure.realtime.RealtimeModel(
voice="en-US-AvaMultilingualNeural",
)
)
await session.start(room=ctx.room, agent=Assistant())
if __name__ == "__main__":
agents.cli.run_app(server)
Advanced Configuration
For full control over Azure Voice Live settings:
import os
from azure.ai.voicelive.models import AzureSemanticVadEn, AudioInputTranscriptionOptions
from livekit.agents import AgentSession
from livekit.plugins import azure
# Configure Semantic VAD for intelligent turn detection
# Options:
# - AzureSemanticVad: Default semantic VAD (multilingual)
# - AzureSemanticVadEn: English-only, optimized for English
# - AzureSemanticVadMultilingual: Explicit multilingual support
turn_detection = AzureSemanticVadEn(
threshold=0.5, # Voice activity detection threshold (0.0-1.0)
silence_duration_ms=500, # Silence duration before turn ends
prefix_padding_ms=300, # Audio padding before speech
speech_duration_ms=200, # Minimum speech duration to trigger detection
remove_filler_words=True, # Remove filler words like "um", "uh"
)
# Configure input audio transcription with language constraint
# This helps prevent language misidentification
# Use azure-speech with text models such as gpt-4o, whisper-1 with gpt-realtime or gpt-realtime-mini
input_audio_transcription = AudioInputTranscriptionOptions(
model="azure-speech",
language="en-US", # Constrain to English for reliable detection
)
session = AgentSession(
llm=azure.realtime.RealtimeModel(
endpoint=os.getenv("AZURE_VOICE_LIVE_ENDPOINT"),
api_key=os.getenv("AZURE_VOICE_LIVE_API_KEY"),
model=os.getenv("AZURE_VOICE_LIVE_MODEL", "gpt-4o"),
voice=os.getenv("AZURE_VOICE_LIVE_VOICE", "en-US-AvaMultilingualNeural"),
input_audio_transcription=input_audio_transcription,
turn_detection=turn_detection,
tool_choice="auto", # Enable function calling
)
)
RealtimeModel Parameters
| Parameter | Description | Default |
|---|---|---|
endpoint |
Azure Voice Live endpoint URL | AZURE_VOICE_LIVE_ENDPOINT env var |
api_key |
Azure API key | AZURE_VOICE_LIVE_API_KEY env var |
use_default_credential |
Authenticate with DefaultAzureCredential instead of an API key |
False |
model |
Model name | AZURE_VOICE_LIVE_MODEL env var, or gpt-realtime |
voice |
Azure neural voice name | en-US-AvaMultilingualNeural |
modalities |
Output modalities, ["text"] for text-only responses |
["text", "audio"] |
input_audio_transcription |
Audio transcription config (model, language), None to disable |
azure-speech for non-multimodal models (e.g. gpt-4.1) and phi4-mm-realtime, whisper-1 for the other models, with auto-detect |
turn_detection |
VAD configuration object | ServerVad(threshold=0.5) |
tool_choice |
Function calling mode ("auto", "none", etc.) | "auto" |
temperature |
Sampling temperature | 0.8 |
max_output_tokens |
Maximum output tokens per response | 4096 |
Turn Detection Options
| VAD Type | Description |
|---|---|
AzureSemanticVad |
Default semantic VAD (multilingual) |
AzureSemanticVadEn |
English-only, optimized for English |
AzureSemanticVadMultilingual |
Explicit multilingual support |
VAD Parameters
| Parameter | Description | Range |
|---|---|---|
threshold |
Voice activity detection sensitivity | 0.0-1.0 |
silence_duration_ms |
Silence duration before turn ends | milliseconds |
prefix_padding_ms |
Audio padding before detected speech | milliseconds |
speech_duration_ms |
Minimum speech duration to trigger | milliseconds |
remove_filler_words |
Remove "um", "uh", etc. | boolean |
Azure Voice Live Capabilities
- Low-latency conversations - Realtime bidirectional streaming
- Multilingual support - Works with Azure's multilingual neural voices
- Function calling - Built-in tool use for agentic workflows
- Interruption handling - Graceful handling of user interruptions
Pipeline Mode (STT + LLM + TTS)
Azure Credentials
Azure Speech Services (for STT/TTS)
export AZURE_SPEECH_KEY=<your-speech-key>
export AZURE_SPEECH_REGION=<your-region> # e.g., eastus, westus2
Azure OpenAI (for LLM)
export AZURE_OPENAI_ENDPOINT=https://<your-resource>.openai.azure.com
export AZURE_OPENAI_API_KEY=<your-openai-key>
export AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4o
export AZURE_OPENAI_API_VERSION=2024-10-01-preview
Quick Start
For more control over individual components:
from livekit.agents import Agent, AgentSession
from livekit.plugins import azure, openai, silero
session = AgentSession(
stt=azure.STT(),
llm=openai.LLM.with_azure(
model="gpt-4o",
azure_endpoint="https://your-resource.openai.azure.com",
api_key="your-api-key",
api_version="2024-10-01-preview",
),
tts=azure.TTS(voice="en-US-JennyNeural"),
vad=silero.VAD.load(),
)
await session.start(room=ctx.room, agent=Agent(instructions="You are helpful."))
Voice Selection
Azure provides a wide variety of neural voices. Use multilingual voices for multi-language support:
# Multilingual voices (recommended)
model = azure.realtime.RealtimeModel(
voice="en-US-AvaMultilingualNeural"
)
# Or use TTS with specific voices
tts = azure.TTS(voice="en-US-JennyNeural")
Popular voices:
en-US-AvaMultilingualNeural- Multilingual, feminineen-US-AndrewMultilingualNeural- Multilingual, masculineen-US-JennyNeural- English, feminineen-US-GuyNeural- English, masculine
See Azure Voice Gallery for full list.
HD Voices
For higher quality audio, use DragonHD voices:
tts = azure.TTS(voice="en-US-Ava:DragonHDLatestNeural")
Resources
Metadata
Release files for livekit-plugins-azure 1.8.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| livekit_plugins_azure-1.8.6.tar.gz | 32.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| livekit_plugins_azure-1.8.6-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 67.8 kB
Release files / livekit_plugins_azure-1.8.6.tar.gz
| Download URL | livekit_plugins_azure-1.8.6.tar.gz |
|---|---|
| Size | 32.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8e80bc2d84fc4313ea2ea0e63f9dddd07ae965068f54b5d8219a5229e65d82b4
|
|
BLAKE2b-256 checksum How to use checksums |
b5cb1fd6e98ad48dd6c1e3851f777c556498189b3f4b23bd5e513155af29cef8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 9, 2026.
Transparency logRelease files / livekit_plugins_azure-1.8.6-py3-none-any.whl
| Download URL | livekit_plugins_azure-1.8.6-py3-none-any.whl |
|---|---|
| Size | 35.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b75197c3b600f72b149e4f356b00fabddd46bc9476adf49d896f74bee98c4242
|
|
BLAKE2b-256 checksum How to use checksums |
d46336d8503b3df69c1377d5da9ddb7ebe05fdcf661ad4dcd82f5026eef9ce3f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 9, 2026.
Transparency log
