arize-ax-cli 0.7.1

Official Arize CLI tool for managing datasets, experiments, and more

---

Arize AX CLI

• Features
• Installation
  • Using pip
  • From source
  • Verify Installation
• Quick Start
  • 1. Initialize Configuration
  • 2. Verify Configuration
  • 3. Start Using the CLI
• Configuration
  • Configuration Commands
  • Configuration Modes
    • Simple Configuration (Recommended)
    • Advanced Configuration
  • Configuration File Location
  • Updating a Profile
  • Configuration Reference
    • All Available Sections
  • Using Environment Variables
    • 1. Auto-Detection During Setup
    • 2. Manual Environment Variable References
    • Viewing Expanded Values
  • Multiple Profiles
• Shell Autocompletion
  • Quick Install (Recommended)
  • Verify Installation
  • Manual Installation (Alternative)
  • Supported Shells
• Commands
  • Global Options
  • AI Integrations
  • Annotation Configs
  • API Keys
  • Cache
  • Datasets
  • Evaluators
  • Experiments
  • Projects
  • Prompts
  • Spans
  • Tasks
  • Traces
• Usage Examples
  • Creating a Dataset from a CSV File
  • Exporting Dataset List to JSON
  • Exporting Dataset Examples
  • Exporting Experiment Runs
  • Exporting Spans by Trace ID
  • Using a Different Profile for a Command
  • Exporting Spans
  • Listing Traces and Exporting to Parquet
  • Pagination
  • Working with Multiple Environments
  • Filtering Spans by Status
  • Listing Traces in a Time Window
• Advanced Topics
  • Output Formats
  • Programmatic Usage
  • Environment Variables
  • Debugging
• Troubleshooting
  • Configuration Issues
  • Connection Issues
  • Shell Completion Not Working
• Getting Help
  • Command-specific Help
  • Support
• Contributing
• License
• Changelog

Official command-line interface for Arize AI - manage your AI observability resources including datasets, projects, spans, traces, and more.

Features

• Dataset Management: Create, list, update, and delete datasets
• Evaluator Management: Create and manage LLM-as-judge evaluators and their versions
• Experiment Management: Run and analyze experiments on your datasets
• Project Management: Organize your projects
• API Key Management: Create, refresh, and revoke API keys
• AI Integrations: Configure external LLM providers (OpenAI, Anthropic, AWS Bedrock, and more)
• Prompt Management: Create and version prompts with label management
• Spans & Traces: Query and filter LLM spans and traces
• Multiple Profiles: Switch between different Arize environments
• Flexible Output: Export to JSON, CSV, Parquet, or display as tables
• Shell Completion: Tab completion for bash, zsh, and fish
• Rich CLI Experience: Beautiful terminal output with progress indicators

Installation

Using pip

pip install arize-ax-cli

From source

git clone https://github.com/Arize-ai/arize-ax-cli.git
cd arize-ax-cli
pip install -e .

Verify Installation

ax --version

Quick Start

1. Initialize Configuration

The first time you use the CLI, you'll need to create a configuration profile:

ax profiles create

This interactive setup will:

• Detect existing ARIZE_* environment variables and offer to use them
• Guide you through credential setup if no environment variables are found
• Create a configuration profile (default or named)
• Save your preferences for output format, caching, and more

Example output:

     _         _                 _   __  __
    / \   _ __(_)_______        / \  \ \/ /
   / _ \ | '__| |_  / _ \      / _ \  \  /
  / ___ \| |  | |/ /  __/     / ___ \ /  \
 /_/   \_\_|  |_/___\___|    /_/   \_\_/\_\
                  AI Observability Platform

Welcome to Arize AX CLI!
No configuration found. Let's set it up!

Environment Variable Detection

  ✓ Detected ARIZE_API_KEY = ak-2a...FCf

Create config from detected environment variables? [Y/n]: y

? Default output format: table

✓ Configuration saved to profile 'default'

You're ready to go! Try: ax datasets list

2. Verify Configuration

Check your configuration:

ax profiles show

3. Start Using the CLI

List your datasets:

ax datasets list

List your projects:

ax projects list

Export spans from a project:

ax spans export <project> --stdout

List traces in a project:

ax traces list <project-id>

Configuration

The Arize CLI uses a flexible configuration system that supports multiple profiles, environment variables, and two setup modes.

Configuration Commands

Command	Description
ax profiles create [name]	Create a new configuration profile interactively or from flags/file
ax profiles update	Update fields in an existing profile
ax profiles list	List all available profiles
ax profiles show	Display the current profile's configuration
ax profiles use <profile>	Switch to a different profile
ax profiles validate	Check profile for missing or incorrect config
ax profiles delete <profile>	Delete a configuration profile

Configuration Modes

You can also create a profile non-interactively using CLI flags or a TOML file:

# Create with flags (no prompts)
ax profiles create staging --api-key ak_abc123 --region US --output-format json

# Create from a TOML file
ax profiles create production --from-file ./prod.toml

# Create from file and override the API key
ax profiles create production --from-file ./prod.toml --api-key ak_override

Flag precedence (highest to lowest): CLI flags → --from-file (TOML) → interactive prompts.

When you run ax profiles create without flags, you'll be prompted to choose between two configuration modes:

Simple Configuration (Recommended)

Best for: Most users, cloud deployments, standard Arize usage

The simple setup only asks for the essentials:

• API Key: Your Arize API key
• Region: US, EU, or leave unset (auto-detect)
• Output Format: table, json, csv, or parquet

Example:

Choose configuration mode:
> Simple (recommended)
  Advanced

API Key: Insert value
API Key (e.g., ak-123...): [hidden input]

Region:
> (leave empty for unset)
  US
  EU
  Use environment variable

Default output format:
> table
  json
  csv
  parquet

Generated configuration:

[profile]
name = "default"

[auth]
api_key = "ak_your_api_key_here"

[routing]
region = "US"

[output]
format = "table"

Advanced Configuration

Best for: On-premise deployments, Private Connect, custom routing, performance tuning

The advanced setup provides full control over:

1. API Key: Your Arize credentials
2. Routing: Choose from multiple strategies:
   • No override (use defaults)
   • Region-based routing (US, EU)
   • Single endpoint (on-premise deployments)
   • Base domain (Private Connect)
   • Custom endpoints & ports (granular control)
3. Transport: Performance tuning:
   • Stream max workers
   • Stream max queue bound
   • PyArrow max chunksize
   • Max HTTP payload size
4. Security: TLS certificate verification
5. Output Format: Default display format

Example routing options:

What type of override should we setup?
  0 - No override (use defaults)
  1 - Region (for region-based routing)
  2 - Single endpoint (typical for on-prem deployments)
> 3 - Base Domain (for Private Connect)
  4 - Custom endpoints & ports

Generated configuration (example with Private Connect):

[profile]
name = "production"

[auth]
api_key = "${ARIZE_API_KEY}"

[routing]
base_domain = "arize-private.yourcompany.com"

[transport]
stream_max_workers = 8
stream_max_queue_bound = 5000
pyarrow_max_chunksize = 10000
max_http_payload_size_mb = 8

[security]
request_verify = true

[storage]
directory = "~/.arize"
cache_enabled = true

[output]
format = "json"

Configuration File Location

Configuration files are stored at:

Profile	Linux/macOS	Windows
default	~/.arize/config.toml	%USERPROFILE%\.arize\config.toml
Named profiles	~/.arize/profiles/<profile>.toml	%USERPROFILE%\.arize\profiles\<profile>.toml

Updating a Profile

Use ax profiles update to modify specific fields in an existing profile without recreating it:

# Update the API key in the active profile
ax profiles update --api-key ak_new_key

# Update the region in a named profile
ax profiles update --profile production --region EU

# Replace an entire profile from a TOML file
ax profiles update --profile production --from-file ./prod.toml

# Load from file and override the API key
ax profiles update --profile staging --from-file ./staging.toml --api-key ak_override

Options:

Option	Description
--profile, -p	Profile to update (uses active profile if omitted)
--from-file, -f	TOML file to load; completely replaces the existing profile
--api-key	Arize API key
--region	Routing region (e.g. us-east-1b, US, EU)
--output-format	Default output format (table, json, csv, parquet)
--verbose, -v	Enable verbose logs

With flags only, just the specified fields are updated; all others are preserved. With --from-file, the profile is fully replaced by the file contents (flags are still applied on top).

Configuration Reference

All Available Sections

Authentication (required)

[auth]
api_key = "ak_your_api_key_here"
# Or use environment variable reference:
api_key = "${ARIZE_API_KEY}"

Routing (choose one strategy)

[routing]
# Option 1: Region-based (recommended for cloud)
region = "US"  # or "EU"

# Option 2: Single endpoint (on-premise)
single_host = "arize.yourcompany.com"
single_port = "443"

# Option 3: Base domain (Private Connect)
base_domain = "arize-private.yourcompany.com"

# Option 4: Custom endpoints (advanced)
api_host = "api.arize.com"
api_scheme = "https"
otlp_host = "otlp.arize.com"
otlp_scheme = "https"
flight_host = "flight.arize.com"
flight_port = "443"
flight_scheme = "grpc+tls"

Transport (optional, advanced only)

[transport]
stream_max_workers = 8
stream_max_queue_bound = 5000
pyarrow_max_chunksize = 10000
max_http_payload_size_mb = 8

Security (optional, advanced only)

[security]
request_verify = true  # Set to false to disable SSL verification (not recommended)

Storage (optional)

[storage]
directory = "~/.arize"
cache_enabled = true

Output (optional)

[output]
format = "table"  # Options: table, json, csv, parquet

Using Environment Variables

The CLI can detect and use environment variables in two ways:

1. Auto-Detection During Setup

When you run ax profiles create, the CLI automatically detects existing ARIZE_* environment variables and offers to use them:

ax profiles create

Environment Variable Detection

  ✓ Detected ARIZE_API_KEY = ak_***************xyz
  ✓ Detected ARIZE_REGION = US

Create profiles from detected environment variables? [Y/n]: y

This will create a configuration that references the environment variables:

[auth]
api_key = "${ARIZE_API_KEY}"

[routing]
region = "${ARIZE_REGION}"

2. Manual Environment Variable References

During both Simple and Advanced setup, you can choose "Use environment variable" for any field to reference an environment variable:

API Key:
  Insert value
> Use environment variable

Environment variable name for API Key: ARIZE_API_KEY

Viewing Expanded Values

To see the actual values (with environment variables expanded):

ax profiles show --expand

Without --expand, you'll see the variable references like ${ARIZE_API_KEY}.

Multiple Profiles

Create different profiles for different environments:

# Create a production profile (name as argument skips the name prompt)
ax profiles create production

# Create a staging profile interactively
ax profiles create staging

# List all profiles
ax profiles list

# Switch profiles
ax profiles use production
ax profiles use staging

# Update a field in a specific profile
ax profiles update --profile staging --region EU

# Use a specific profile for a single command
ax datasets list --profile production

# Delete a profile
ax profiles delete staging

Shell Autocompletion

Enable tab completion for your shell to autocomplete commands, options, and arguments.

Quick Install (Recommended)

The CLI includes a built-in installer that automatically configures completion for your shell:

ax --install-completion

This will:

• Detect your current shell (bash, zsh, or fish)
• Install the appropriate completion script
• Show you instructions to activate it

After running the command, restart your shell or open a new terminal window for the changes to take effect.

Verify Installation

Once installed, test tab completion:

ax <TAB>         # Shows available commands (cache, datasets, experiments, profiles, projects, spans, traces)
ax datasets <TAB> # Shows dataset subcommands (list, get, export, create, append, delete)
ax datasets list --<TAB>  # Shows available options

Manual Installation (Alternative)

If you prefer to see or customize the completion script before installing:

# View the completion script for your shell
ax --show-completion

# Save it to a file and source it manually
ax --show-completion >> ~/.bashrc  # For bash
ax --show-completion >> ~/.zshrc   # For zsh

Supported Shells

• Bash (Linux, macOS, Windows Git Bash)
• Zsh (macOS default, Oh My Zsh)
• Fish (Linux, macOS)
• PowerShell (Windows)

Commands

Global Options

Available for all commands:

• --profile, -p <name>: Use a specific configuration profile
• --output, -o <format>: Set output format (table, json, csv, parquet, or a file path)
• --help, -h: Show help message

Note: --verbose, -v is available on each individual subcommand (e.g., ax datasets list --verbose) rather than as a top-level flag.

AI Integrations

Configure external LLM providers for use within the Arize platform (for evaluations, online evals, and more):

# List AI integrations
ax ai-integrations list [--space-id <space-id>] [--limit 15] [--cursor <cursor>]

# Get a specific integration
ax ai-integrations get <integration-id>

# Create an integration (OpenAI example)
ax ai-integrations create --name "OpenAI Prod" --provider openAI \
  --api-key <key> --model-name gpt-4o --model-name gpt-4o-mini

# Create an integration with custom headers
ax ai-integrations create --name "Custom LLM" --provider custom \
  --base-url https://my-llm.example.com \
  --headers-json '{"X-API-Key": "secret"}'

# Create an AWS Bedrock integration
ax ai-integrations create --name "Bedrock" --provider awsBedrock \
  --provider-metadata-json '{"role_arn": "arn:aws:iam::123456789:role/MyRole"}'

# Update an integration
ax ai-integrations update <integration-id> --name "Renamed" --api-key <new-key>

# Delete an integration
ax ai-integrations delete <integration-id> [--force]

Supported providers:

Provider	Value	Notes
OpenAI	openAI
Azure OpenAI	azureOpenAI	Use --base-url for the deployment endpoint
AWS Bedrock	awsBedrock	Requires --provider-metadata-json
Vertex AI	vertexAI	Requires --provider-metadata-json
Anthropic	anthropic
NVIDIA NIM	nvidiaNim
Google Gemini	gemini
Custom	custom	Use --base-url for a custom endpoint

Annotation Configs

Manage annotation configs (rubrics for human and automated evaluation):

# List annotation configs
ax annotation-configs list [--space-id <space-id>] [--limit 15] [--cursor <cursor>]

# Get a specific annotation config
ax annotation-configs get <annotation-config-id>

# Create a freeform annotation config (free-text feedback)
ax annotation-configs create --name "Quality" --space-id <space-id> --type freeform

# Create a continuous annotation config (numeric score range)
ax annotation-configs create --name "Score" --space-id <space-id> --type continuous \
  --min-score 0 --max-score 1 --optimization-direction maximize

# Create a categorical annotation config (discrete labels)
ax annotation-configs create --name "Verdict" --space-id <space-id> --type categorical \
  --value good --value neutral --value bad --optimization-direction maximize

# Delete an annotation config
ax annotation-configs delete <annotation-config-id> [--force]

Supported annotation config types:

Type	Required options	Optional options
freeform	(none)	—
continuous	--min-score, --max-score	--optimization-direction
categorical	--value (repeat for multiple labels, e.g. --value good --value bad)	--optimization-direction

API Keys

Security note: The raw key value is only returned once (on create and refresh). Store it securely immediately — it cannot be retrieved again.

# List API keys
ax api-keys list [--key-type user|service] [--status active|deleted] \
  [--limit 15] [--cursor <cursor>]

# Create a user key (authenticates as you)
ax api-keys create --name "My Key" [--description "..."] [--expires-at 2025-12-31T23:59:59]

# Create a service key (scoped to a space)
ax api-keys create --name "CI Key" --key-type service --space-id <space-id>

# Refresh a key (revokes old key, issues replacement)
ax api-keys refresh <key-id> [--expires-at 2025-12-31T23:59:59]

# Delete a key
ax api-keys delete <key-id> [--force]

Key types:

Type	Description
user	Authenticates as the creating user. Global, so --space-id not needed.
service	Scoped to a specific space. --space-id is required.

Cache

Manage the local cache. The CLI caches downloaded resource data (e.g., dataset examples) locally as Parquet files to avoid redundant API calls. When you fetch a dataset's examples, the results are stored on disk so subsequent requests for the same version load instantly. The cache is automatically invalidated when a resource's updated_at timestamp changes, so you always get fresh data when something changes on the server.

Caching is enabled by default and can be toggled in your profile configuration:

[storage]
cache_enabled = true

# Clear the cache
ax cache clear

Datasets

Manage your datasets:

# List datasets
ax datasets list --space-id <space-id> [--limit 15] [--cursor <cursor>]

# Get dataset metadata
ax datasets get <dataset-id>

# Export all examples to a file
ax datasets export <dataset-id> [--version-id <version-id>] [--output-dir .] [--stdout]

# Create a new dataset
ax datasets create --name "My Dataset" --space-id <space-id> --file data.csv

# Append examples (inline JSON)
ax datasets append <dataset-id> --json '[{"question": "...", "answer": "..."}]'

# Append examples (from file)
ax datasets append <dataset-id> --file new_examples.csv [--version-id <version-id>]

# Delete a dataset
ax datasets delete <dataset-id> [--force]

Supported data file formats:

• CSV (.csv)
• JSON (.json)
• JSON Lines (.jsonl)
• Parquet (.parquet)

Evaluators

Manage LLM-as-judge evaluators and their versions:

# List evaluators (optionally filtered by space)
ax evaluators list [--space-id <space-id>] [--limit 15] [--cursor <cursor>]

# Get an evaluator with its latest version
ax evaluators get <evaluator-id>

# Get an evaluator at a specific version
ax evaluators get <evaluator-id> --version-id <version-id>

# Create a new evaluator
ax evaluators create \
  --name "Response Relevance" \
  --space-id <space-id> \
  --commit-message "Initial version" \
  --template-name relevance \
  --template "Is this response relevant to the query? {{input}} {{output}}" \
  --ai-integration-id <integration-id> \
  --model-name gpt-4o

# Create a classification evaluator (label → numeric score; omit flag for freeform)
ax evaluators create \
  --name "Relevance classifier" \
  --space-id <space-id> \
  --commit-message "Initial version" \
  --template-name relevance \
  --template "Classify: {{output}}" \
  --ai-integration-id <integration-id> \
  --model-name gpt-4o \
  --classification-choices '{"relevant":1,"irrelevant":0}' \
  --direction maximize \
  --data-granularity span

# Update evaluator metadata
ax evaluators update <evaluator-id> --name "New Name"
ax evaluators update <evaluator-id> --description "Updated description"

# Delete an evaluator (and all its versions)
ax evaluators delete <evaluator-id> [--force]

# List all versions of an evaluator
ax evaluators list-versions <evaluator-id> [--limit 15] [--cursor <cursor>]

# Get a specific version by ID
ax evaluators get-version <version-id>

# Create a new version of an existing evaluator
ax evaluators create-version <evaluator-id> \
  --commit-message "Improved prompt" \
  --template-name relevance \
  --template "Rate the relevance of the response: {{input}} {{output}}" \
  --ai-integration-id <integration-id> \
  --model-name gpt-4o

# Same optional template fields as create (e.g. classification choices)
ax evaluators create-version <evaluator-id> \
  --commit-message "Add rails" \
  --template-name relevance \
  --template "Classify: {{output}}" \
  --ai-integration-id <integration-id> \
  --model-name gpt-4o \
  --classification-choices '{"relevant":1,"irrelevant":0}'

Template configuration options:

Option	Description
--template-name	Eval column name (alphanumeric, spaces, hyphens, underscores)
--template	Prompt template with {{variable}} placeholders referencing span attributes
--ai-integration-id	AI integration global ID (base64)
--model-name	Model name (e.g. gpt-4o, claude-3-5-sonnet)
--include-explanations	Include reasoning explanation alongside the score (default: on)
--use-function-calling	Prefer structured function-call output when supported (default: on)
--invocation-params	JSON object of model invocation parameters (e.g. '{"temperature": 0.7}')
--provider-params	JSON object of provider-specific parameters
--classification-choices	JSON object mapping labels to numeric scores (e.g. '{"relevant":1,"irrelevant":0}'); omit for freeform output
--direction	maximize or minimize (optimization direction for scores)
--data-granularity	span, trace, or session

Experiments

Run and analyze experiments on your datasets:

# List experiments (optionally filtered by dataset)
ax experiments list [--dataset-id <dataset-id>] [--limit 15] [--cursor <cursor>]

# Get a specific experiment
ax experiments get <experiment-id>

# Export all runs from an experiment
ax experiments export <experiment-id> [--output-dir .] [--stdout]

# Create a new experiment from a data file
ax experiments create --name "My Experiment" --dataset-id <dataset-id> --file runs.csv

# List runs for an experiment
ax experiments list_runs <experiment-id> [--limit 30]

# Delete an experiment
ax experiments delete <experiment-id> [--force]

Note: The data file for experiments create must contain example_id and output columns. Extra columns are passed through as additional fields.

Export options:

Option	Description
--output-dir	Output directory (default: current directory)
--stdout	Print JSON to stdout instead of saving to file
--profile, -p	Configuration profile to use
--verbose, -v	Enable verbose logs

Projects

Organize your projects:

# List projects
ax projects list --space-id <space-id> [--limit 15] [--cursor <cursor>]

# Get project metadata
ax projects get <project-id>

# Create a new project
ax projects create --name "My Project" --space-id <space-id>

# Delete a project
ax projects delete <project-id> [--force]

Prompts

Manage versioned prompt templates with label-based deployment:

# List prompts
ax prompts list [--space-id <space-id>] [--limit 15] [--cursor <cursor>]

# Get a prompt (latest version by default)
ax prompts get <prompt-id>

# Get a specific version by ID or label
ax prompts get <prompt-id> --version-id <version-id>
ax prompts get <prompt-id> --label production

# Create a prompt with an initial version
ax prompts create \
  --name "My Prompt" \
  --space-id <space-id> \
  --provider openAI \
  --input-variable-format f_string \
  --messages messages.json \
  --commit-message "Initial version"

# Update a prompt's description
ax prompts update <prompt-id> --description "Updated description"

# Delete a prompt (removes all versions)
ax prompts delete <prompt-id> [--force]

# List versions for a prompt
ax prompts list-versions <prompt-id> [--limit 15] [--cursor <cursor>]

# Create a new version
ax prompts create-version <prompt-id> \
  --provider openAI \
  --input-variable-format f_string \
  --messages messages_v2.json \
  --commit-message "Improved system prompt"

# Create a new version (inline messages JSON)
ax prompts create-version <prompt-id> \
  --provider openAI \
  --input-variable-format f_string \
  --messages '[{"role": "user", "content": "Your prompt here"}]' \
  --commit-message "Minimal inline JSON example"


# Resolve a label to its version
ax prompts get-version-by-label <prompt-id> --label production

# Set labels on a version (replaces all existing labels)
ax prompts set-version-labels <version-id> --label production --label staging

# Remove a label from a version
ax prompts remove-version-label <version-id> --label staging

Messages (--messages): pass a path to a JSON file, or inline JSON. Inline values must start with [ or { after whitespace (so a missing file path like msgs.json yields a clear “file not found” error instead of a JSON parse error). The payload must be a non-empty JSON array of message objects. Example file messages.json:

[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user",   "content": "Summarize the following: {text}"},
  {"role": "assistant", "tool_calls": [{"id": "tool-call-1", "type": "function", "function": {"name": "search", "arguments": "{\"query\": \"summarize {text}\"}}]},
  {"role": "tool", "tool_call_id": "tool-call-1", "content": "This is the result of the search function."},
]

Input variable formats:

Format	Syntax
f_string	{variable_name}
mustache	{{variable_name}}
none	No variable parsing

Spans

Export LLM spans from a project. Spans are individual units of work (e.g., an LLM call, a tool call) within a trace. By default spans are written to a JSON file; use --stdout to print to stdout instead.

# Export all spans (writes to file by default)
ax spans export <project>

# Export with filter
ax spans export <project> --filter "status_code = 'ERROR'"

# Export by trace, span, or session ID
ax spans export <project> --trace-id <trace-id>
ax spans export <project> --span-id <span-id>
ax spans export <project> --session-id <session-id>

# Export to stdout
ax spans export <project> --stdout

Options:

Option	Description
--trace-id	Filter by trace ID
--span-id	Filter by span ID
--session-id	Filter by session ID
--filter	Filter expression (e.g. status_code = 'ERROR', latency_ms > 1000)
--space-id	Space ID (required when using a project name instead of ID)
--limit, -n	Maximum number of spans to export (default: 100)
--days	Lookback window in days (default: 30)
--start-time	Override start of time window (ISO 8601)
--end-time	Override end of time window (ISO 8601)
--output-dir	Output directory (default: current directory)
--stdout	Print JSON to stdout instead of saving to file
--profile, -p	Configuration profile to use
--verbose, -v	Enable verbose logs

Examples:

ax spans export <project> --filter "status_code = 'ERROR'"
ax spans export <project> --filter "latency_ms > 1000"
ax spans export <project> --trace-id abc123 --filter "latency_ms > 1000"
ax spans export <project> --start-time 2024-01-01T00:00:00Z --end-time 2024-01-02T00:00:00Z

Tasks

Manage evaluation tasks and trigger on-demand runs:

# List tasks (optionally filtered by space, project, dataset, or type)
ax tasks list [--space-id <space-id>] [--project-id <project-id>] \
  [--dataset-id <dataset-id>] [--task-type template_evaluation|code_evaluation] \
  [--limit 15] [--cursor <cursor>]

# Get a specific task
ax tasks get <task-id>

# Create a project-based task
ax tasks create \
  --name "Relevance Check" \
  --task-type template_evaluation \
  --evaluators '[{"evaluator_id": "<evaluator-id>"}]' \
  --project-id <project-id> \
  --is-continuous

# Create a dataset-based task
ax tasks create \
  --name "Dataset Eval" \
  --task-type template_evaluation \
  --evaluators '[{"evaluator_id": "<evaluator-id>"}]' \
  --dataset-id <dataset-id> \
  --experiment-ids <exp-id-1>,<exp-id-2>

# Trigger an on-demand run
ax tasks trigger-run <task-id>

# Trigger a run and wait for it to complete
ax tasks trigger-run <task-id> --wait

# Trigger a run over a specific data window
ax tasks trigger-run <task-id> \
  --data-start-time 2024-01-01T00:00:00Z \
  --data-end-time 2024-01-02T00:00:00Z \
  --max-spans 5000

# List runs for a task (optionally filtered by status)
ax tasks list-runs <task-id> [--status pending|running|completed|failed|cancelled] \
  [--limit 15] [--cursor <cursor>]

# Get a specific run
ax tasks get-run <run-id>

# Cancel a run (only valid when pending or running)
ax tasks cancel-run <run-id> [--force]

# Wait for a run to reach a terminal state
ax tasks wait-for-run <run-id> [--poll-interval 5] [--timeout 600]

create options:

Option	Description
--name, -n	Task name (must be unique within the space)
--task-type	template_evaluation or code_evaluation
--evaluators	JSON array of evaluator objects: [{"evaluator_id": "...", "query_filter": null, "column_mappings": null}]
--project-id	Project global ID (base64); mutually exclusive with --dataset-id
--dataset-id	Dataset global ID (base64); mutually exclusive with --project-id
--experiment-ids	Comma-separated experiment IDs (required for dataset-based tasks)
--sampling-rate	Fraction of data to evaluate (0–1); project tasks only
--is-continuous / --no-continuous	Run continuously on incoming data
--query-filter	Task-level filter applied to all evaluators

trigger-run options:

Option	Description
--data-start-time	ISO 8601 start of the data window
--data-end-time	ISO 8601 end of the data window (defaults to now)
--max-spans	Maximum spans to evaluate (default: 10 000)
--override-evaluations	Re-evaluate data that already has labels
--experiment-ids	Comma-separated experiment IDs; dataset-based tasks only
--wait, -w	Block until the run reaches a terminal state
--poll-interval	Seconds between polling attempts when --wait is set (default: 5)
--timeout	Maximum seconds to wait when --wait is set (default: 600)

Traces

Query traces in a project. A trace is a collection of spans representing a full request or conversation; the CLI identifies traces by their root span (parent_id = null). The CLI automatically applies parent_id = null; any --filter you provide is ANDed with it.

# List traces
ax traces list <project-id> [--start-time <iso8601>] [--end-time <iso8601>] \
  [--filter "<expr>"] [--limit 15] [--cursor <cursor>] [--output <format>]

Options:

Option	Description
--start-time	Start of time window, inclusive (ISO 8601, e.g. 2024-01-01T00:00:00Z)
--end-time	End of time window, exclusive (ISO 8601). Defaults to now
--filter	Filter expression (e.g. status_code = 'ERROR', latency_ms > 1000)
--limit, -n	Maximum number of traces to return (default: 15)
--cursor	Pagination cursor for the next page
--output, -o	Output format (table, json, csv, parquet) or file path
--profile, -p	Configuration profile to use
--verbose, -v	Enable verbose logs

Filter examples:

ax traces list <project-id> --filter "status_code = 'ERROR'"
ax traces list <project-id> --start-time 2024-01-01T00:00:00Z
ax traces list <project-id> --filter "latency_ms > 5000" --limit 50

Usage Examples

Creating a Dataset from a CSV File

ax datasets create \
  --name "Customer Churn Dataset" \
  --space-id sp_abc123 \
  --file ./data/churn.csv

Exporting Dataset List to JSON

ax datasets list --space-id sp_abc123 --output json > datasets.json

Exporting Dataset Examples

# Export to a timestamped directory
ax datasets export ds_xyz789

# Export a specific version
ax datasets export ds_xyz789 --version-id ver_abc123

# Pipe to jq for processing
ax datasets export ds_xyz789 --stdout | jq '.[].input'

Exporting Experiment Runs

# Export all runs to a timestamped directory
ax experiments export exp_abc123

# Pipe to stdout for processing
ax experiments export exp_abc123 --stdout | jq '.[] | select(.output != null)'

Exporting Spans by Trace ID

# Export all spans in a trace
ax spans export proj_abc123 --trace-id tr_xyz789

# Export a session's spans to stdout
ax spans export proj_abc123 --session-id sess_456 --stdout

# Export with a custom lookback window
ax spans export proj_abc123 --trace-id tr_xyz789 --days 7

Using a Different Profile for a Command

ax datasets list --space-id sp_abc123 --profile production

Exporting Spans

# Export all spans from a project
ax spans export proj_abc123

# Export error spans
ax spans export proj_abc123 --filter "status_code = 'ERROR'" --limit 100

# Export spans in a time window to stdout
ax spans export proj_abc123 --start-time 2024-01-01T00:00:00Z --end-time 2024-01-02T00:00:00Z --stdout

Listing Traces and Exporting to Parquet

# List root traces in a project
ax traces list proj_abc123

# Export slow traces to Parquet for analysis
ax traces list proj_abc123 --filter "latency_ms > 2000" --limit 500 --output traces_slow.parquet

# List traces in JSON format
ax traces list proj_abc123 --output json

Pagination

List more datasets using pagination:

# First page
ax datasets list --space-id sp_abc123 --limit 20

# Next page (use cursor from previous response)
ax datasets list --space-id sp_abc123 --limit 20 --cursor <cursor-value>

Working with Multiple Environments

# Setup profiles for different environments
ax profiles create  # Create "production" profile
ax profiles create  # Create "staging" profile

# Switch contexts
ax profiles use production
ax datasets list --space-id sp_prod123

ax profiles use staging
ax datasets list --space-id sp_stage456

Filtering Spans by Status

ax spans export <project> --filter "status_code = 'ERROR'" --stdout

Listing Traces in a Time Window

ax traces list <project-id> \
  --start-time 2024-01-01T00:00:00Z \
  --end-time 2024-01-02T00:00:00Z

Advanced Topics

Output Formats

The CLI supports multiple output formats:

1. Table (default): Human-readable table format
2. JSON: Machine-readable JSON
3. CSV: Comma-separated values
4. Parquet: Apache Parquet columnar format

Set default format in profiles:

ax profiles create  # Select output format during setup

Or override per command:

ax datasets list --output json
ax datasets list --output datasets.csv
ax datasets list --output datasets.parquet

Programmatic Usage

Integrate with scripts:

#!/bin/bash

# Export datasets to JSON
DATASETS=$(ax datasets list --space-id sp_abc123 --output json)

# Process with jq
echo "$DATASETS" | jq '.data[] | select(.name | contains("test"))'

# Export to file
ax datasets export ds_xyz789

Environment Variables

The CLI respects these environment variables:

• ARIZE_API_KEY: Your Arize API key
• ARIZE_REGION: Region (US, EU, etc.)
• Any other ARIZE_* variables will be detected during ax profiles create

Debugging

Enable verbose mode to see detailed SDK logs:

ax datasets list --space-id sp_abc123 --verbose

Troubleshooting

Configuration Issues

Problem: Profile 'default' not found.

Solution: Run ax profiles create to create a configuration profile.

---

Problem: Invalid API key

Solution: Verify your API key:

1. Check your configuration: ax profiles show
2. Refresh your API key from the Arize UI
3. Update your profile: ax profiles update --api-key <new-key>

---

Connection Issues

Problem: Connection refused or SSL errors

Solution:

1. Check your routing configuration: ax profiles show
2. Verify network connectivity
3. For on-premise installations, ensure single_host is configured correctly
4. For SSL issues, check security.request_verify setting (use with caution)

---

Shell Completion Not Working

Problem: Tab completion doesn't work

Solution:

1. Verify completion is installed: Run the installation command for your shell
2. Reload your shell or open a new terminal
3. Ensure ax is in your PATH: which ax

---

Getting Help

Command-specific Help

Every command has detailed help:

ax --help
ax datasets --help
ax datasets create --help
ax profiles --help

Support

• Documentation: https://arize.com/docs/api-clients/cli/
• Bug Reports: GitHub Issues
• Community: Arize Community Slack
• Email: [e-mail]

Contributing

We welcome contributions!

• For developers: See DEVELOPMENT.md for architecture, code structure, and development guide
• For contributors: See CONTRIBUTING.md for contribution guidelines (coming soon)

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Changelog

See CHANGELOG.md for release notes and version history.

---

Built with ❤️ by Arize AI