Microsoft Teams Bridge for ElevenLabs Agents (Python)

Bridge Microsoft Teams voice/video calls to an ElevenLabs Agent.

PyPI package: elevenlabs-msteams-bridge - the -py suffix is only in this repository's name, to distinguish it from the Node.js sibling repo.

This is the Python sibling of @komaa/elevenlabs-msteams-bridge (Node.js) - same wire contract, same environment variables, drop-in interchangeable behind the same .env file. The Node and Python packages version independently, so a version-number gap between them is expected; both implement the same StandIn wire protocol and interoperate with the hosted service identically. It terminates the StandIn media bridge wire protocol on one side and the ElevenLabs Agent WebSocket on the other:

• No transcoding: both sides speak base64 PCM 16 kHz mono - the hot path is copy-only.
• Barge-in: ElevenLabs interruptions map to playback flushes, with ghost-audio filtering.
• On-demand vision: the agent's look client tool answers from the caller's camera or screen-share, via your own OpenAI-compatible vision endpoint (frames never persisted) or via ElevenLabs multimodal upload (gated on Teams recording).
• Call governors: a bridge-side hard time cap with a deterministic TTS goodbye, plus the worker-side governor.
• Hardened: HMAC-signed upgrades with replay guard, connection caps, SSRF-guarded image fetches, dead-peer detection, graceful SIGTERM drain, Prometheus /metrics.

StandIn is the hosted media bridge that joins the Teams call and dials this bridge - you run no Teams media stack yourself.

Documentation: komaa-com.github.io/elevenlabs-msteams-bridge-py (getting started, example walkthrough, configuration and library reference, wire protocol). Teams/StandIn setup lives at docs.komaa.com.

Install

pip install elevenlabs-msteams-bridge

Requires Python 3.10+.

Run

ELEVENLABS_API_KEY=sk_... \
ELEVENLABS_AGENT_ID=agent_... \
WORKER_SHARED_SECRET=... \
elevenlabs-msteams-bridge

A .env file in the working directory is loaded automatically (existing environment wins). The bridge listens on ws://0.0.0.0:8080/voice/msteams/stream by default; StandIn appends /{callId} per call. Expose the port with a tunnel and register the wss:// URL as your identity's Agent voice URL in the StandIn dashboard.

Your ElevenLabs agent's audio input and output format must be PCM 16000 Hz - the bridge ends the call with a clear error if the agent negotiates anything else.

Embed

import asyncio
from elevenlabs_msteams_bridge import load_config, start_server

async def main():
    server = await start_server(load_config())
    await asyncio.Event().wait()  # run until cancelled

asyncio.run(main())

Pass your own async vision callable to answer the agent's look tool with any model you like - the raw frame never leaves your process:

async def describe(frame: dict, question: str) -> str:
    ...  # call your vision model with frame["dataBase64"] / frame["mime"]
    return "a person holding a badge"

server = await start_server(load_config(), vision=describe)

Configuration

Everything is environment variables; names are identical to the Node package.

Endpoints

• GET /healthz - liveness.
• GET /metrics - Prometheus counters (calls, rejections, relayed/dropped frames).
• GET /{...}/{callId} + WebSocket upgrade - the worker wire, HMAC-signed with X-OpenClawTeamsBridge-Timestamp / X-OpenClawTeamsBridge-Signature over "{timestampMs}.{callId}".

Notes for operators:

• /healthz and /metrics are unauthenticated (only the WebSocket upgrade is HMAC-gated). They expose no call content - just liveness and counters - but if you would rather not leak call volumes, keep the port behind your ingress/tunnel rules.
• One bridge process serves one agent id (ELEVENLABS_AGENT_ID). Run one process per agent if you route multiple agents.

Vision and recording

The look tool prefers your VISION_API_URL endpoint: the frame is described transiently and only the text enters the conversation. Without one, the bridge falls back to uploading the frame to ElevenLabs (multimodal) - that persists the frame with a third party, so it is only allowed while Teams recording is active. Note that even path-2 descriptions become ElevenLabs conversation content, which ElevenLabs retains per your agent's settings; enable the agent's zero-retention mode if callers' surroundings must not be stored.

License

MIT (c) Komaa DigiTech