unmute-core

unmute-core is a Python library for voice agents on LiveKit Agents and Pipecat. You keep your framework, your speech and model plugins, and your pipeline. The library adds what a real phone or web call needs on top: typed call state, task steps, tools, handoffs, idle callers, transfers and tracing.

Unmute compiles an agent spec into a LiveKit or Pipecat project. Those projects import this library. You can also use it in an agent you write by hand, one piece at a time or all of it.

Example

A plain LiveKit agent with one tool from the library:

import os

from livekit.agents import Agent, AgentSession
from livekit.plugins import openai, silero, slng
from pydantic import BaseModel

from unmute_core.livekit import function_tools
from unmute_core.tools import ToolRegistry, local_tool


class OrderStatus(BaseModel):
    order_id: str
    status: str


def order_status(order_id: str) -> OrderStatus:
    """Look up an order by its id. The return annotation is the result's checked shape."""
    return OrderStatus(order_id=order_id, status="out for delivery")


registry = ToolRegistry([local_tool(order_status, announce="One moment, let me check.")])

agent = Agent(instructions="You help with orders.", tools=function_tools(registry))
slng_host = f"{os.environ['SLNG_WORLD_PART']}.api.slng.ai"
stt = slng.STT(slng_base_url=slng_host, model="soniox/speech-ai:rt-v5")
session = AgentSession(
    stt=stt,
    llm=openai.LLM(),
    tts=slng.TTS(
        slng_base_url=slng_host,
        model="slng/fish/tts:s2.1-pro",
        voice="8d8aec63a56c460cb7def795bdf754fa",
    ),
    vad=silero.VAD.load(),
)
stt.attach_to_session(session)

The agent stays a normal LiveKit Agent. The library adds the tool and the line the agent says while the tool runs. Speech runs on SLNG: Soniox hears the caller and Fish Audio speaks, with your SLNG_API_KEY. On Pipecat, put tools_schema(registry) on the LLMContext instead.

How it fits

flowchart TD
    U[Unmute compiles agent.yaml] --> P[Your LiveKit or Pipecat project]
    H[You write agent.py by hand] --> P
    P -->|builds the pipeline, calls| A[unmute_core.livekit or unmute_core.pipecat]
    A --> C[Core: state, tasks, tools, hooks, handoff, telemetry, model call]

1. Your project builds the pipeline. Unmute writes that project for you, or you write it yourself.
2. The project calls an adapter: unmute_core.livekit or unmute_core.pipecat.
3. The adapter turns framework events into calls to the core. The core never imports LiveKit or Pipecat.
4. The same core runs on both frameworks. Shared test scenarios run on both adapters and must give the same result.

The library never reads agent.yaml and never builds the pipeline. Your speech-to-text, text-to-speech and LLM plugins stay yours.

Three ways to use it

An agent can also be defined as data: state as type text like Phone | None, tasks and tools as JSON. See examples/parcel-desk.

Four whole agents show other shapes of call, each on LiveKit and Pipecat from one package, on SLNG speech and Gemini in the EU: a restaurant finder with tools only (nosh), a car claim intake (kestrel), an outbound payment reminder (wasla) and a bilingual language tutor (faro). See example agents and type the path.

Why use it

LiveKit and Pipecat move audio, run the models and call your tools. They do not decide what a good phone call looks like. Every team learns that from its first real calls. unmute-core holds what we learned, so you start from it.

1. One agent, two frameworks. State, tasks, tools, prompts and traces run the same on LiveKit and Pipecat. 90 test scenarios run on both frameworks, with plain and realtime models, and must give the same result.
2. The model cannot break your data. Each value the model saves is checked against its type. A wrong value is refused with a reason the model can act on, like caller_phone: expected a phone number with its country code. The model asks the caller again.
3. Over 100 call optimizations, built in. Some examples:

Why unmute-core follows one call from start to end. Each feature page then starts with what it fixes.

Install

Python 3.12 or newer. Pick the extra for your framework:

uv add "unmute-core[livekit]"

uv add "unmute-core[pipecat]"

Everything works with your framework's own plugins and no SLNG package. installation covers version ranges and the version check at import.

Docs

The docs tell one story. Start small, then go deeper where you need to.

1. Get started: why unmute-core, installation and the quickstart.
2. Build a salon agent: a tutorial that grows one agent, one piece per page. It ends at examples/salon-concierge-unmute.
3. On your framework: add one piece to your agent, or run a whole agent on LiveKit or Pipecat.
4. Guides: state, tools, knowledge, tasks, handoffs, call lifecycle, prefetch, turn taking, hooks, model call, slng, realtime, telephony and telemetry.
5. Example agents: overview, type the path, and one page each for nosh, kestrel, wasla and faro.
6. Reference: settings and tasks.

Versions

The library is 0.x. Code written for 0.14.x runs on any later 0.14.x. A new minor version, like 0.15.0, may break it. unmute_core.require() checks this when your code imports.

CHANGELOG.md lists what each release changed, and what can break you first.

Develop

uv sync --all-extras
uv run pytest
uv run ruff check . && uv run ruff format --check .
uvx ty check

CONTRIBUTING.md explains how to send a change. SECURITY.md explains how to report a security problem.

License

MIT. See LICENSE.