voicetest

Open-source test harness for voice agent workflows.

Simulate conversations — LLM-powered users talk to your agent, LLM judges score the results
Test any platform — Retell, VAPI, LiveKit, Bland, Telnyx, or custom agents
Convert between formats — Import from one platform, export to another via a unified graph IR
Diagnose failures — Auto-fix broken prompts with an LLM-powered repair loop
Run anywhere — CLI, Web UI, REST API, CI/CD

Installation

uv tool install voicetest

Or add to a project (uv run voicetest to run):

uv add voicetest

Or with pip:

pip install voicetest

Quick start

Try voicetest with a sample healthcare receptionist agent and 8 test cases:

# Set up an API key (free, no credit card at https://console.groq.com)
export GROQ_API_KEY=gsk_...

# Load demo and start web UI
voicetest demo --serve

Tip: If you have Claude Code installed, skip API key setup and use claudecode/sonnet as your model. See Claude Code Passthrough.

Web UI Demo (light)

Web UI

voicetest serve

Agent import, graph visualization, test execution with real-time streaming transcripts, run history, diagnosis, and more at http://localhost:8000.

Platform support

Import from any supported format, convert through the unified AgentGraph, and export to any other:

Retell CF ─────┐                  ┌───▶ Retell LLM
               │                  │
Retell LLM ────┼                  ├───▶ Retell CF
               │                  │
VAPI ──────────┼                  ├───▶ VAPI
               │                  │
Bland ─────────┼───▶ AgentGraph ──┼───▶ Bland
               │                  │
Telnyx ────────┤                  ├───▶ Telnyx
               │                  │
LiveKit ───────┤                  ├───▶ LiveKit
               │                  │
XLSForm ───────┤                  ├───▶ Mermaid
               │                  │
Custom ────────┘                  └───▶ Voicetest JSON

Platform	Import	Push	Sync	API Key Env Var
Retell	✓	✓	✓	`RETELL_API_KEY`
VAPI	✓	✓	✓	`VAPI_API_KEY`
Bland	✓	✓		`BLAND_API_KEY`
Telnyx	✓	✓	✓	`TELNYX_API_KEY`
LiveKit	✓	✓	✓	`LIVEKIT_API_KEY` + `LIVEKIT_API_SECRET`

CI/CD

Run voice agent tests in GitHub Actions to catch regressions before production:

name: Voice Agent Tests
on:
  push:
    paths: ["agents/**"]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v5
      - run: uv tool install voicetest
      - run: voicetest run --agent agents/receptionist.json --tests agents/tests.json --all
        env:
          GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}

Configuration

Settings are stored in .voicetest/settings.toml:

[models]
agent = "groq/llama-3.1-8b-instant"
simulator = "groq/llama-3.1-8b-instant"
judge = "groq/llama-3.1-8b-instant"

[run]
max_turns = 20
audio_eval = false
streaming = false

Any LiteLLM-compatible model works — OpenAI, Anthropic, Google, Ollama, and more. See the full configuration reference.

Documentation

Full documentation is at voicetest.dev/docs.

Topic	Description
Getting Started	Install, demo, first test walkthrough
Core Concepts	Agent graphs, node types, test cases
CLI Reference	All commands and options
Features	Format conversion, diagnosis, audio eval, snippets, and more
Configuration	Models, settings, Claude Code, platform credentials
Architecture	Engine internals, DI, storage
Development	Contributing, Docker setup, code quality

Contact

Questions, feedback, or partnerships: hello@voicetest.dev

License

Apache 2.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

voicetest-0.50-py3-none-any.whl (1.5 MB view details)

Uploaded Jul 20, 2026 Python 3

File details

Details for the file voicetest-0.50-py3-none-any.whl.

File metadata

Download URL: voicetest-0.50-py3-none-any.whl
Upload date: Jul 20, 2026
Size: 1.5 MB
Tags: Python 3
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for voicetest-0.50-py3-none-any.whl
Algorithm	Hash digest
SHA256	`e674a7bbee7e5bbba1872128ea5d0cb716259c2311a397a100ab3f97133c38e3`
MD5	`5b3197d8224d72ecda23266902fe7d07`
BLAKE2b-256	`b36e991c19b5fdf3789a2d79bcc3ce722b047ac9fe951cbdb6bee4a50861d6dd`

See more details on using hashes here.

Provenance

The following attestation bundles were made for voicetest-0.50-py3-none-any.whl:

Publisher: publish.yml on voicetestdev/voicetest

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: voicetest-0.50-py3-none-any.whl
- Subject digest: e674a7bbee7e5bbba1872128ea5d0cb716259c2311a397a100ab3f97133c38e3
- Sigstore transparency entry: 2206910495
- Sigstore integration time: Jul 20, 2026
Source repository:
- Permalink: voicetestdev/voicetest@dae1ad7be2b622e8c846982273132e24da176724
- Branch / Tag: refs/heads/main
- Owner: https://github.com/voicetestdev
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@dae1ad7be2b622e8c846982273132e24da176724
- Trigger Event: workflow_run