TestMu Agent Testing CLI

TestMu Agent Testing CLI is a command-line tool for testing chat, voice, and Phone Caller agents from your terminal. Discover projects and test resources, run evaluations, retrieve results, and automate test runs in CI/CD.

Requirements

• Python 3.10 or newer
• A TestMu account with access to Agent Testing

Install

Install the latest published version:

pip install agent-testing-cli

Check the available commands:

agent-testing-cli --help

Authenticate

Before you sign in, see Get started with TestMu Automation to obtain your username and access key.

Sign in locally. Enter your TestMu username and access key when prompted:

agent-testing-cli login

Check sign-in status:

agent-testing-cli auth status

For CI/CD, store these values as protected secrets and set both environment variables:

export LT_USERNAME="YOUR_USERNAME"
export LT_ACCESS_KEY="YOUR_ACCESS_KEY"

Discover Test Resources

agent-testing-cli projects
agent-testing-cli --project PROJECT_ID scenarios
agent-testing-cli --project PROJECT_ID suites
agent-testing-cli --project PROJECT_ID endpoint-profiles

Use --json when a script or CI/CD job reads the output:

agent-testing-cli --json projects

Run Tests

For a Phone Caller run, --project PROJECT_ID identifies the TestMu project and --suite SUITE_ID identifies the suite to run:

agent-testing-cli --project PROJECT_ID run --suite SUITE_ID

Use --yes to confirm a run without an interactive prompt:

agent-testing-cli --project PROJECT_ID run --suite SUITE_ID --yes

Run a Chat evaluation:

agent-testing-cli --project PROJECT_ID run \
  --workflow WORKFLOW_ID \
  --suite SUITE_ID \
  --yes

Do not use --wait with a Chat evaluation.

Run a suite from a supported TestMu result URL:

agent-testing-cli run --url "SUITE_URL" --yes

Run multiple suite URLs in one command:

agent-testing-cli run --url "SUITE_URL_1" "SUITE_URL_2" "SUITE_URL_3" --yes

Retrieve Results

agent-testing-cli results --suite SUITE_ID
agent-testing-cli call --id CALL_ID

Interactive Terminal Interface

Open the interactive terminal user interface (TUI):

agent-testing-cli tui

Use agent-testing-cli --help for the complete command and option reference.