rtvoice 0.4.0

Real-time voice assistant built on OpenAI's Realtime API

rtvoice

A Python library for building real-time voice agents powered by the OpenAI Realtime API. It handles the full session lifecycle — microphone input, WebSocket streaming, turn detection, tool calling, and audio playback — so you can focus on what your agent does, not how it talks.

---

Installation

pip install rtvoice[audio]

Requires Python 3.13+ and an OPENAI_API_KEY environment variable (or pass api_key= directly).

---

Quickstart

import asyncio
from rtvoice import RealtimeAgent

async def main():
    agent = RealtimeAgent(
        instructions="You are Jarvis, a concise and helpful voice assistant.",
    )
    await agent.run()

asyncio.run(main())

Run it, speak into your microphone, and the agent responds through your speakers. Press Ctrl+C to end the session.

---

Tool calling

Register any async (or sync) function with @tools.action(...) and the model will call it when appropriate:

import asyncio
from typing import Annotated
from rtvoice import RealtimeAgent, Tools

tools = Tools()

@tools.action("Get the current weather for a given city")
async def get_weather(city: Annotated[str, "The city name"]) -> str:
    return f"It's 18°C and partly cloudy in {city}."

async def main():
    agent = RealtimeAgent(
        instructions="Answer weather questions using get_weather.",
        tools=tools,
    )
    await agent.run()

asyncio.run(main())

For long-running tools, set is_long_running=True and provide a holding_instruction so the assistant keeps the user informed while it works. → Tools guide

---

Supervisor agents

Delegate complex, multi-step tasks to a dedicated LLM-driven sub-agent. The voice agent hands off the task, speaks a holding phrase, and presents the result when done:

from llmify import ChatOpenAI
from rtvoice import RealtimeAgent, SupervisorAgent, Tools

tools = Tools()

@tools.action("Book a restaurant table.")
async def book_table(restaurant: str, date: str, time: str, party_size: int) -> str:
    return f"Booked table for {party_size} at {restaurant} on {date} at {time}."

booking_agent = SupervisorAgent(
    name="Booking Assistant",
    description="Books restaurant tables for the user.",
    holding_instruction="I'm checking availability, just a moment.",
    instructions="Use book_table to complete booking requests.",
    tools=tools,
    llm=ChatOpenAI(model="gpt-4o-mini"),
)

agent = RealtimeAgent(
    instructions="Delegate restaurant bookings to the Booking Assistant.",
    supervisor_agent=booking_agent,
)

If the supervisor needs information from the user (e.g. party size), it asks a clarifying question through the voice agent automatically. → Supervisor guide

---

MCP servers

Connect any MCP-compatible tool server via MCPServerStdio. Tools are discovered and registered automatically during prepare():

from rtvoice import RealtimeAgent
from rtvoice.mcp import MCPServerStdio

agent = RealtimeAgent(
    instructions="You can read and write files in /tmp.",
    mcp_servers=[
        MCPServerStdio(
            command="npx",
            args=["-y", "@modelcontextprotocol/server-filesystem", "/tmp"],
        )
    ],
)

Prefer attaching MCP servers to a SupervisorAgent rather than RealtimeAgent directly to keep the realtime model's tool list short. → MCP guide

---

Custom audio devices

Implement AudioInputDevice or AudioOutputDevice to use any audio source or sink — useful for testing, telephony, or embedded hardware:

from collections.abc import AsyncIterator
from rtvoice.audio import AudioInputDevice

class CustomMicrophone(AudioInputDevice):
    async def start(self) -> None: ...
    async def stop(self) -> None: ...

    async def stream_chunks(self) -> AsyncIterator[bytes]:
        while self.is_active:
            yield await self._read_audio_chunk()

    @property
    def is_active(self) -> bool:
        return self._active

agent = RealtimeAgent(
    instructions="...",
    audio_input=CustomMicrophone(),
)

→ Audio API reference

---

Documentation

Full documentation including guides and API reference: mathisarends.github.io/rtvoice

• Quickstart
• Tools
• Supervisor Agent
• MCP Servers
• Listener
• API Reference