veris-cli 1.15.0

CLI to connect local agents to the Veris backend

veris-cli

Veris CLI connects a local agent to the Veris simulation backend so you can generate scenarios and run end-to-end evaluations.

Quickstart

Before you start

• Python 3.11+
• uv installed

If you want to run simulationa against you local agent you need ngrok setup: Follow: https://ngrok.com/docs/getting-started/

You need to have an API key and Agent ID from the Veris team.

Installation

uv add tool veris-cli

Initialize (one time only)

IMPORTANT: Get your VERIS_API_KEY and VERIS_AGENT_ID if you don't have it already. Ask us on Slack or email [e-mail] if need help.

veris init --veris-api-key YOUR_API_KEY --veris-agent-id YOUR_AGENT_ID

This creates .veris/config.yaml with your credentials.

Commands

Local Setup

If you're running a local agent, use ngrok to create a public tunnel. If the agent is being served on a different pathname than root add the --agent-pathname parameter:

# Start tunnel in background (default)
veris setup-local start --local-url http://localhost:8000

# Start tunnel with custom agent pathname like /mcp or /ai/mcp (for MCP agents)
veris setup-local start --local-url http://localhost:8000 --agent-pathname /mcp

# Run in foreground (blocks until Ctrl+C)
veris setup-local start --local-url http://localhost:8000 --foreground

# Stop all tunnels
veris setup-local stop

This saves the public URL to .veris/config.yaml under PUBLIC_AGENT_URL.

Agent Management

# Create a new agent from a natural language prompt
veris agent create --prompt "A customer support agent that helps users with billing inquiries"

# Create with a specific version
veris agent create --prompt "A travel booking assistant" --version v2.0.0

# Show agent information
veris agent show

# Or specify a different agent
veris agent show --agent-id AGENT_ID

Scenario Management

# Create a new scenario set
veris scenario create --num-scenarios 10

# Create for a specific agent version
veris scenario create --num-scenarios 10 --version-id VERSION_ID

# List scenario sets for your agent
veris scenario list

# Or specify a different agent
veris scenario list --agent-id AGENT_ID

Simulation

# Launch simulation
veris simulation launch --scenario-set-id SET_ID --max-turns 20 --watch

# Launch with custom agent connection settings
veris simulation launch \
  --scenario-set-id SET_ID \
  --max-turns 20 \
  --agent-url https://your-agent.com \
  --agent-transport http \
  --agent-timeout 300 \
  --watch

# Get simulation status
veris simulation status --run-id RUN_ID

# Get sessions
veris simulation sessions --run-id RUN_ID

# Get logs (all sessions or specific session)
veris simulation logs --run-id RUN_ID
veris simulation logs --run-id RUN_ID --session-id SESSION_ID

# Get results
veris simulation results --run-id RUN_ID

# Kill running simulation
veris simulation kill --run-id RUN_ID

Configuration

Credentials and settings are stored in .veris/config.yaml:

VERIS_API_KEY: your_api_key
VERIS_AGENT_ID: your_agent_id
VERIS_API_URL: https://simulator.api.veris.ai/
PUBLIC_AGENT_URL: https://xxxx.ngrok.io  # Set by setup-local

Development

# Install dependencies
uv sync

# Run tests
uv run pytest

# Format code
uv run ruff format

# Lint
uv run ruff check --fix