voice-agent

Reusable Python voice-agent components with Kokoro and Piper TTS, local STT, and an asynchronous OpenRouter LLM adapter. A turn-based conversation session connects these adapters.

Setup

Use uv and Python 3.12 or newer. Add only the providers you need to a consuming project, for example:

uv add "voice-agent-56[openrouter,stt]"

From this repository checkout, install the TTS providers with:

uv sync --locked --extra kokoro --extra piper

Select one extra to install only that provider. Add dependencies with uv add and commit pyproject.toml with uv.lock. Run scripts with uv run.

TTS

from voice_agent.tts import create

engine = create("piper:en_US-lessac-medium", models_dir="/path/to/models")
for audio, sample_rate in engine.stream("Hello. How can I help?"):
    print(len(audio), sample_rate)

Chunks contain mono float32 NumPy audio at the returned sample rate. Streaming is sentence-based. The application owns playback and resampling.

Supply model files through models_dir; they are not stored in Git:

• Kokoro needs kokoro-v1.0.onnx and voices-v1.0.bin. Its default voice is af_heart.
• Piper needs <voice>.onnx and <voice>.onnx.json. Its default voice is en_US-lessac-medium.

STT

Install local recognition with uv sync --locked --extra stt. The adapters accept mono float32 samples at 16 kHz and return a Transcript. The application owns decoding, resampling, and recording limits.

from voice_agent.stt import create

engine = create("whisper", models_dir="models/stt", num_threads=2)
result = engine.transcribe(samples, language="en")
print(result.text)

Choose whisper for multilingual Whisper base or zipformer for English. Whisper also accepts language=None for detection. Zipformer accepts only English. Calls return completed-utterance results. They do not provide a live microphone API. Whisper includes segment timestamps; Zipformer currently returns text without segment timestamps. Callers must serialize access to each engine.

Model directories under models_dir are whisper-base and zipformer-en-full. The checked-out model assets are in models/stt/. See STT research and local measurements for the exact models, versions, limitations, and benchmark command. Model files stay outside Git.

LLM through OpenRouter

In this repository checkout, install the provider and configure your key:

uv sync --locked --extra openrouter
cp .env.example .env
# Set OPENROUTER_API_KEY in .env, then run:
uv run --env-file .env --extra openrouter python scripts/chat_llm.py

The default model is stealth/space-bunny-alpha. This is the model described in the supplied AI/ML API documentation, and OpenRouter lists the same model ID. Requests go directly to https://openrouter.ai/api/v1/chat/completions and require an OpenRouter key, not an AI/ML API key. Set OPENROUTER_MODEL to change models, or pass model= explicitly. The library reads environment variables; the uv run --env-file option loads the example configuration for the script.

import asyncio
from contextlib import aclosing
from voice_agent.llm import create

async def main():
    messages = [
        {"role": "system", "content": "Reply briefly in natural spoken language. Avoid markdown."},
        {"role": "user", "content": "Hello. Can you help me?"},
    ]
    async with create("openrouter") as engine:
        async with aclosing(engine.stream(messages)) as response:
            async for text in response:
                print(text, end="", flush=True)
        # Or await engine.complete(messages) for one complete string.

asyncio.run(main())

stream() yields text deltas, which may be fragments of words. Buffer them into sentences before passing them to TTS. The application owns the system prompt, conversation history, sentence buffering, and playback. This adapter supports text messages with system, user, and assistant roles; tool execution and full audio conversation orchestration are not implemented.

Provider options are api_key, model, max_tokens with a default of 2048, reasoning_effort with a default of low, and timeout with a default of 60 seconds of network inactivity. The token budget also covers reasoning. Use a larger budget if the provider hits its limit. Set reasoning_effort=None to omit the reasoning option for another model. Only answer content is yielded.

Use aclosing around a stream if you might stop early. Cancelling the task or closing the iterator closes its HTTP response. Use the engine as an async context manager, or call await engine.aclose() when finished. Requests are not retried automatically. LLMError reports HTTP failures, timeouts, stream errors, truncation, and empty answers. Partial text may already have been yielded when an error occurs; the application decides what to play or retain in history.

Run the provider tests without credentials or model downloads:

uv run --locked --extra openrouter python -m unittest discover -s tests -v

Conversation sessions

voice_agent.session.ConversationSession provides turn-based orchestration and isolated in-memory history. Supply a frozen SessionConfig, an asynchronous LLM adapter, and async transcription and synthesis callbacks. The caller owns file decoding, native-model concurrency, transport, deadlines, and session expiry.

Call begin() before consuming turn(audio, generation). It emits stage, transcript, text delta, WAV audio, and completion events. Call acknowledge with the generation only after playback completes. Cancel invalidates old generations and closes active LLM work. It does not stop synchronous native inference that is already running. Interrupted replies are marked in history.

This first session implementation uses completed English utterances and waits for a full generated reply before synthesis. It does not implement streaming microphone input, automatic barge-in, tool calls, or a remote worker protocol. The live integration is in the sibling voice-lab application.

Design

See architecture, provider research, and NVIDIA model notes. Applications supply transport, context, and domain tools. Keep application setup in the consuming repository.

The package is licensed under Apache-2.0.