russo 0.3.3

Testing framework for LLM tool-call accuracy — audio & text

Documentation: https://mohit2152sharma.github.io/russo

Source Code: https://github.com/mohit2152sharma/russo

russo is a testing framework for verifying that LLM agents make the correct tool calls when given audio (or text) input. Think of it as pytest for voice AI tool-calling accuracy.

Key Features

• Provider-agnostic — works with Gemini, OpenAI, or any custom agent via protocols
• Audio-first — synthesize text prompts to audio, send to your agent, evaluate tool calls
• pytest integration — use markers, fixtures, and familiar test patterns
• Built-in caching — skip TTS on repeated runs, saving time and money
• Extensible — swap synthesizers, agents, evaluators, and parsers via structural typing

Quick Start

pip install russo

With optional providers:

pip install "russo[openai]"    # OpenAI support
pip install "russo[ws]"        # WebSocket agents
pip install "russo[all]"       # Everything

Minimal Example

import russo
from russo.synthesizers import GoogleSynthesizer
from russo.adapters import GeminiLiveAgent
from russo.evaluators import ExactEvaluator

result = await russo.run(
    prompt="Book a flight from Berlin to Rome for tomorrow",
    synthesizer=GoogleSynthesizer(api_key="..."),
    agent=GeminiLiveAgent(api_key="...", tools=[...]),
    evaluator=ExactEvaluator(),
    expect=[
        russo.tool_call("book_flight", from_city="Berlin", to_city="Rome"),
    ],
)

russo.assert_tool_calls(result)

pytest Integration

import pytest
import russo

@pytest.mark.russo(
    prompt="Book a flight from NYC to LA",
    expect=[russo.tool_call("book_flight", from_city="NYC", to_city="LA")],
)
async def test_book_flight(russo_result):
    russo.assert_tool_calls(russo_result)

License

This project is licensed under the terms of the MIT license.