voicetest

Open-source test harness for voice agent workflows.

Simulate conversations — LLM-powered users talk to your agent, LLM judges score the results
Test any platform — Retell, VAPI, LiveKit, Bland, Telnyx, or custom agents
Convert between formats — Import from one platform, export to another via a unified graph IR
Diagnose failures — Auto-fix broken prompts with an LLM-powered repair loop
Run anywhere — CLI, Web UI, REST API, CI/CD

Installation

uv tool install voicetest

Or add to a project (uv run voicetest to run):

uv add voicetest

Or with pip:

pip install voicetest

Quick start

Try voicetest with a sample healthcare receptionist agent and 8 test cases:

# Set up an API key (free, no credit card at https://console.groq.com)
export GROQ_API_KEY=gsk_...

# Load demo and start web UI
voicetest demo --serve

Tip: If you have Claude Code installed, skip API key setup and use claudecode/sonnet as your model. See Claude Code Passthrough.

Web UI Demo (light)

Web UI

voicetest serve

Agent import, graph visualization, test execution with real-time streaming transcripts, run history, diagnosis, and more at http://localhost:8000.

Platform support

Import from any supported format, convert through the unified AgentGraph, and export to any other:

Retell CF ─────┐                  ┌───▶ Retell LLM
               │                  │
Retell LLM ────┼                  ├───▶ Retell CF
               │                  │
VAPI ──────────┼                  ├───▶ VAPI
               │                  │
Bland ─────────┼───▶ AgentGraph ──┼───▶ Bland
               │                  │
Telnyx ────────┤                  ├───▶ Telnyx
               │                  │
LiveKit ───────┤                  ├───▶ LiveKit
               │                  │
XLSForm ───────┤                  ├───▶ Mermaid
               │                  │
Custom ────────┘                  └───▶ Voicetest JSON

Platform	Import	Push	Sync	API Key Env Var
Retell	✓	✓	✓	`RETELL_API_KEY`
VAPI	✓	✓	✓	`VAPI_API_KEY`
Bland	✓	✓		`BLAND_API_KEY`
Telnyx	✓	✓	✓	`TELNYX_API_KEY`
LiveKit	✓	✓	✓	`LIVEKIT_API_KEY` + `LIVEKIT_API_SECRET`

CI/CD

Run voice agent tests in GitHub Actions to catch regressions before production:

name: Voice Agent Tests
on:
  push:
    paths: ["agents/**"]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v5
      - run: uv tool install voicetest
      - run: voicetest run --agent agents/receptionist.json --tests agents/tests.json --all
        env:
          GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}

Configuration

Settings are stored in .voicetest/settings.toml:

[models]
agent = "groq/llama-3.1-8b-instant"
simulator = "groq/llama-3.1-8b-instant"
judge = "groq/llama-3.1-8b-instant"

[run]
max_turns = 20
audio_eval = false
streaming = false

Any LiteLLM-compatible model works — OpenAI, Anthropic, Google, Ollama, and more. See the full configuration reference.

Documentation

Full documentation is at voicetest.dev/docs.

Topic	Description
Getting Started	Install, demo, first test walkthrough
Core Concepts	Agent graphs, node types, test cases
CLI Reference	All commands and options
Features	Format conversion, diagnosis, audio eval, snippets, and more
Configuration	Models, settings, Claude Code, platform credentials
Architecture	Engine internals, DI, storage
Development	Contributing, Docker setup, code quality

Contact

Questions, feedback, or partnerships: hello@voicetest.dev

License

Apache 2.0

Metadata

Release files for voicetest 0.51

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for voicetest 0.51
File	Interpreter	ABI	Platform	Reset
voicetest-0.51-py3-none-any.whl 1.5 MB Oct 14, 2026	Python 3	none	any	Details

Release files / voicetest-0.51-py3-none-any.whl

Download URL	voicetest-0.51-py3-none-any.whl
Size	1.5 MB
Tags	Python 3
SHA-256 checksum How to use checksums	`4b1c0780d87f2a82ef56f2ca996871401a7719aaa94ba91ad2d9ad40428488db`
BLAKE2b-256 checksum How to use checksums	`ebbc001218aa616550ca339e1b19df9bf011790e7f8b43de45a61544883febb3`
Upload date	Oct 14, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/voicetestdev/voicetest
Publishing commit: github.com/voicetestdev/voicetest/tree/cd4a677774ac8a79f547fd0d56a00e07bf6b27fe

Workflow

Publishing configuration: github.com/voicetestdev/voicetest/blob/cd4a677774ac8a79f547fd0d56a00e07bf6b27fe/.github/workflows/publish.yml
Publishing logs: github.com/voicetestdev/voicetest/actions/runs/38515743293/attempts/1

Artifact

Subject: voicetest-0.51-py3-none-any.whl
SHA-256 checksum: 4b1c0780d87f2a82ef56f2ca996871401a7719aaa94ba91ad2d9ad40428488db Verifying attestations