arize-ax-cli 0.4.0

Official Arize CLI tool for managing datasets, experiments, and more

---

Arize AX CLI

• Features
• Installation
  • Using pip
  • From source
  • Verify Installation
• Quick Start
  • 1. Initialize Configuration
  • 2. Verify Configuration
  • 3. Start Using the CLI
• Configuration
  • Configuration Commands
  • Configuration Modes
    • Simple Configuration (Recommended)
    • Advanced Configuration
  • Configuration File Location
  • Configuration Reference
    • All Available Sections
  • Using Environment Variables
    • 1. Auto-Detection During Setup
    • 2. Manual Environment Variable References
    • Viewing Expanded Values
  • Multiple Profiles
• Shell Autocompletion
  • Quick Install (Recommended)
  • Verify Installation
  • Manual Installation (Alternative)
  • Supported Shells
• Commands
  • Datasets
  • Experiments
  • Projects
  • Spans
  • Traces
  • Annotation Configs
  • Cache
  • Global Options
• Usage Examples
  • Creating a Dataset from a CSV File
  • Exporting Dataset List to JSON
  • Exporting Dataset Examples
  • Exporting Experiment Runs
  • Exporting Spans by Trace ID
  • Using a Different Profile for a Command
  • Listing Spans and Exporting to CSV
  • Listing Traces and Exporting to Parquet
  • Pagination
  • Working with Multiple Environments
  • Filtering Spans by Status
  • Listing Traces in a Time Window
• Advanced Topics
  • Output Formats
  • Programmatic Usage
  • Environment Variables
  • Debugging
• Troubleshooting
  • Configuration Issues
  • Connection Issues
  • Shell Completion Not Working
• Getting Help
  • Command-specific Help
  • Support
• Contributing
• License
• Changelog

Official command-line interface for Arize AI - manage your AI observability resources including datasets, projects, spans, traces, and more.

Features

• Dataset Management: Create, list, update, and delete datasets
• Experiment Management: Run and analyze experiments on your datasets
• Project Management: Organize your projects
• Spans & Traces: Query and filter LLM spans and traces
• Multiple Profiles: Switch between different Arize environments
• Flexible Output: Export to JSON, CSV, Parquet, or display as tables
• Shell Completion: Tab completion for bash, zsh, and fish
• Rich CLI Experience: Beautiful terminal output with progress indicators

Installation

Using pip

pip install arize-ax-cli

From source

git clone https://github.com/Arize-ai/arize-ax-cli.git
cd arize-ax-cli
pip install -e .

Verify Installation

ax --version

Quick Start

1. Initialize Configuration

The first time you use the CLI, you'll need to create a configuration profile:

ax profiles create

This interactive setup will:

• Detect existing ARIZE_* environment variables and offer to use them
• Guide you through credential setup if no environment variables are found
• Create a configuration profile (default or named)
• Save your preferences for output format, caching, and more

Example output:

     _         _                 _   __  __
    / \   _ __(_)_______        / \  \ \/ /
   / _ \ | '__| |_  / _ \      / _ \  \  /
  / ___ \| |  | |/ /  __/     / ___ \ /  \
 /_/   \_\_|  |_/___\___|    /_/   \_\_/\_\
                  AI Observability Platform

Welcome to Arize AX CLI!
No configuration found. Let's set it up!

Environment Variable Detection

  ✓ Detected ARIZE_API_KEY = ak-2a...FCf

Create config from detected environment variables? [Y/n]: y

? Default output format: table

✓ Configuration saved to profile 'default1'

You're ready to go! Try: ax datasets list

2. Verify Configuration

Check your configuration:

ax profiles show

3. Start Using the CLI

List your datasets:

ax datasets list

List your projects:

ax projects list

Export spans from a project:

ax spans export <project> --stdout

List traces in a project:

ax traces list <project-id>

Configuration

The Arize CLI uses a flexible configuration system that supports multiple profiles, environment variables, and two setup modes.

Configuration Commands

Command	Description
ax profiles create	Create a new configuration profile interactively
ax profiles list	List all available profiles
ax profiles show	Display the current profile's configuration
ax profiles use <profile>	Switch to a different profile
ax profiles validate	Check profile for missing or incorrect config
ax profiles delete <profile>	Delete a configuration profile

Configuration Modes

When you run ax profiles create, you'll be prompted to choose between two configuration modes:

Simple Configuration (Recommended)

Best for: Most users, cloud deployments, standard Arize usage

The simple setup only asks for the essentials:

• API Key: Your Arize API key
• Region: US, EU, or leave unset (auto-detect)
• Output Format: table, json, csv, or parquet

Example:

Choose configuration mode:
> Simple (recommended)
  Advanced

API Key: Insert value
API Key (e.g., ak-123...): [hidden input]

Region:
> (leave empty for unset)
  US
  EU
  Use environment variable

Default output format:
> table
  json
  csv
  parquet

Generated configuration:

[profile]
name = "default"

[auth]
api_key = "ak_your_api_key_here"

[routing]
region = "US"

[output]
format = "table"

Advanced Configuration

Best for: On-premise deployments, Private Connect, custom routing, performance tuning

The advanced setup provides full control over:

1. API Key: Your Arize credentials
2. Routing: Choose from multiple strategies:
   • No override (use defaults)
   • Region-based routing (US, EU)
   • Single endpoint (on-premise deployments)
   • Base domain (Private Connect)
   • Custom endpoints & ports (granular control)
3. Transport: Performance tuning:
   • Stream max workers
   • Stream max queue bound
   • PyArrow max chunksize
   • Max HTTP payload size
4. Security: TLS certificate verification
5. Output Format: Default display format

Example routing options:

What type of override should we setup?
  0 - No override (use defaults)
  1 - Region (for region-based routing)
  2 - Single endpoint (typical for on-prem deployments)
> 3 - Base Domain (for Private Connect)
  4 - Custom endpoints & ports

Generated configuration (example with Private Connect):

[profile]
name = "production"

[auth]
api_key = "${ARIZE_API_KEY}"

[routing]
base_domain = "arize-private.yourcompany.com"

[transport]
stream_max_workers = 8
stream_max_queue_bound = 5000
pyarrow_max_chunksize = 10000
max_http_payload_size_mb = 8

[security]
request_verify = true

[storage]
directory = "~/.arize"
cache_enabled = true

[output]
format = "json"

Configuration File Location

Configuration files are stored at:

Profile	Linux/macOS	Windows
default	~/.arize/config.toml	%USERPROFILE%\.arize\config.toml
Named profiles	~/.arize/profiles/<profile>.toml	%USERPROFILE%\.arize\profiles\<profile>.toml

Configuration Reference

All Available Sections

Authentication (required)

[auth]
api_key = "ak_your_api_key_here"
# Or use environment variable reference:
api_key = "${ARIZE_API_KEY}"

Routing (choose one strategy)

[routing]
# Option 1: Region-based (recommended for cloud)
region = "US"  # or "EU"

# Option 2: Single endpoint (on-premise)
single_host = "arize.yourcompany.com"
single_port = "443"

# Option 3: Base domain (Private Connect)
base_domain = "arize-private.yourcompany.com"

# Option 4: Custom endpoints (advanced)
api_host = "api.arize.com"
api_scheme = "https"
otlp_host = "otlp.arize.com"
otlp_scheme = "https"
flight_host = "flight.arize.com"
flight_port = "443"
flight_scheme = "grpc+tls"

Transport (optional, advanced only)

[transport]
stream_max_workers = 8
stream_max_queue_bound = 5000
pyarrow_max_chunksize = 10000
max_http_payload_size_mb = 8

Security (optional, advanced only)

[security]
request_verify = true  # Set to false to disable SSL verification (not recommended)

Storage (optional)

[storage]
directory = "~/.arize"
cache_enabled = true

Output (optional)

[output]
format = "table"  # Options: table, json, csv, parquet

Using Environment Variables

The CLI can detect and use environment variables in two ways:

1. Auto-Detection During Setup

When you run ax profiles create, the CLI automatically detects existing ARIZE_* environment variables and offers to use them:

ax profiles create

Environment Variable Detection

  ✓ Detected ARIZE_API_KEY = ak_***************xyz
  ✓ Detected ARIZE_REGION = US

Create profiles from detected environment variables? [Y/n]: y

This will create a configuration that references the environment variables:

[auth]
api_key = "${ARIZE_API_KEY}"

[routing]
region = "${ARIZE_REGION}"

2. Manual Environment Variable References

During both Simple and Advanced setup, you can choose "Use environment variable" for any field to reference an environment variable:

API Key:
  Insert value
> Use environment variable

Environment variable name for API Key: ARIZE_API_KEY

Viewing Expanded Values

To see the actual values (with environment variables expanded):

ax profiles show --expand

Without --expand, you'll see the variable references like ${ARIZE_API_KEY}.

Multiple Profiles

Create different profiles for different environments:

# Create a production profile
ax profiles create
# Enter profile name: production

# Create a staging profile
ax profiles create
# Enter profile name: staging

# List all profiles
ax profiles list

# Switch profiles
ax profiles use production
ax profiles use staging

# Use a specific profile for a single command
ax datasets list --profile production

# Delete a profile
ax profiles delete production

Shell Autocompletion

Enable tab completion for your shell to autocomplete commands, options, and arguments.

Quick Install (Recommended)

The CLI includes a built-in installer that automatically configures completion for your shell:

ax --install-completion

This will:

• Detect your current shell (bash, zsh, or fish)
• Install the appropriate completion script
• Show you instructions to activate it

After running the command, restart your shell or open a new terminal window for the changes to take effect.

Verify Installation

Once installed, test tab completion:

ax <TAB>         # Shows available commands (cache, datasets, experiments, profiles, projects, spans, traces)
ax datasets <TAB> # Shows dataset subcommands (list, get, export, create, append, delete)
ax datasets list --<TAB>  # Shows available options

Manual Installation (Alternative)

If you prefer to see or customize the completion script before installing:

# View the completion script for your shell
ax --show-completion

# Save it to a file and source it manually
ax --show-completion >> ~/.bashrc  # For bash
ax --show-completion >> ~/.zshrc   # For zsh

Supported Shells

• Bash (Linux, macOS, Windows Git Bash)
• Zsh (macOS default, Oh My Zsh)
• Fish (Linux, macOS)
• PowerShell (Windows)

Commands

Datasets

Manage your datasets:

# List datasets
ax datasets list --space-id <space-id> [--limit 15] [--cursor <cursor>]

# Get dataset metadata
ax datasets get <dataset-id>

# Export all examples to a file
ax datasets export <dataset-id> [--version-id <version-id>] [--output-dir .] [--stdout]

# Create a new dataset
ax datasets create --name "My Dataset" --space-id <space-id> --file data.csv

# Append examples (inline JSON)
ax datasets append <dataset-id> --json '[{"question": "...", "answer": "..."}]'

# Append examples (from file)
ax datasets append <dataset-id> --file new_examples.csv [--version-id <version-id>]

# Delete a dataset
ax datasets delete <dataset-id> [--force]

Supported data file formats:

• CSV (.csv)
• JSON (.json)
• JSON Lines (.jsonl)
• Parquet (.parquet)

Experiments

Run and analyze experiments on your datasets:

# List experiments (optionally filtered by dataset)
ax experiments list [--dataset-id <dataset-id>] [--limit 15] [--cursor <cursor>]

# Get a specific experiment
ax experiments get <experiment-id>

# Export all runs from an experiment
ax experiments export <experiment-id> [--output-dir .] [--stdout]

# Create a new experiment from a data file
ax experiments create --name "My Experiment" --dataset-id <dataset-id> --file runs.csv

# List runs for an experiment
ax experiments list_runs <experiment-id> [--limit 30]

# Delete an experiment
ax experiments delete <experiment-id> [--force]

Note: The data file for experiments create must contain example_id and output columns. Extra columns are passed through as additional fields.

Export options:

Option	Description
--output-dir	Output directory (default: current directory)
--stdout	Print JSON to stdout instead of saving to file
--profile, -p	Configuration profile to use
--verbose, -v	Enable verbose logs

Projects

Organize your projects:

# List projects
ax projects list --space-id <space-id> [--limit 15] [--cursor <cursor>]

# Get project metadata
ax projects get <project-id>

# Create a new project
ax projects create --name "My Project" --space-id <space-id>

# Delete a project
ax projects delete <project-id> [--force]

Spans

Export LLM spans from a project. Spans are individual units of work (e.g., an LLM call, a tool call) within a trace. By default spans are written to a JSON file; use --stdout to print to stdout instead.

# Export all spans (writes to file by default)
ax spans export <project>

# Export with filter
ax spans export <project> --filter "status_code = 'ERROR'"

# Export by trace, span, or session ID
ax spans export <project> --trace-id <trace-id>
ax spans export <project> --span-id <span-id>
ax spans export <project> --session-id <session-id>

# Export to stdout
ax spans export <project> --stdout

Options:

Option	Description
--trace-id	Filter by trace ID
--span-id	Filter by span ID
--session-id	Filter by session ID
--filter	Filter expression (e.g. status_code = 'ERROR', latency_ms > 1000)
--space-id	Space ID (required when using a project name instead of ID)
--limit, -n	Maximum number of spans to export (default: 100)
--days	Lookback window in days (default: 30)
--start-time	Override start of time window (ISO 8601)
--end-time	Override end of time window (ISO 8601)
--output-dir	Output directory (default: current directory)
--stdout	Print JSON to stdout instead of saving to file
--profile, -p	Configuration profile to use
--verbose, -v	Enable verbose logs

Examples:

ax spans export <project> --filter "status_code = 'ERROR'"
ax spans export <project> --filter "latency_ms > 1000"
ax spans export <project> --trace-id abc123 --filter "latency_ms > 1000"
ax spans export <project> --start-time 2024-01-01T00:00:00Z --end-time 2024-01-02T00:00:00Z

Traces

Query traces in a project. A trace is a collection of spans representing a full request or conversation; the CLI identifies traces by their root span (parent_id = null). The CLI automatically applies parent_id = null; any --filter you provide is ANDed with it.

# List traces
ax traces list <project-id> [--start-time <iso8601>] [--end-time <iso8601>] \
  [--filter "<expr>"] [--limit 15] [--cursor <cursor>] [--output <format>]

Options:

Option	Description
--start-time	Start of time window, inclusive (ISO 8601, e.g. 2024-01-01T00:00:00Z)
--end-time	End of time window, exclusive (ISO 8601). Defaults to now
--filter	Filter expression (e.g. status_code = 'ERROR', latency_ms > 1000)
--limit, -n	Maximum number of traces to return (default: 15)
--cursor	Pagination cursor for the next page
--output, -o	Output format (table, json, csv, parquet) or file path
--profile, -p	Configuration profile to use
--verbose, -v	Enable verbose logs

Filter examples:

ax traces list <project-id> --filter "status_code = 'ERROR'"
ax traces list <project-id> --start-time 2024-01-01T00:00:00Z
ax traces list <project-id> --filter "latency_ms > 5000" --limit 50

Annotation Configs

Manage annotation configs (rubrics for human and automated evaluation):

# List annotation configs
ax annotation-configs list [--space-id <space-id>] [--limit 15] [--cursor <cursor>]

# Get a specific annotation config
ax annotation-configs get <annotation-config-id>

# Create a freeform annotation config (free-text feedback)
ax annotation-configs create --name "Quality" --space-id <space-id> --type freeform

# Create a continuous annotation config (numeric score range)
ax annotation-configs create --name "Score" --space-id <space-id> --type continuous \
  --min-score 0 --max-score 1 --optimization-direction maximize

# Create a categorical annotation config (discrete labels)
ax annotation-configs create --name "Verdict" --space-id <space-id> --type categorical \
  --value good --value neutral --value bad --optimization-direction maximize

# Delete an annotation config
ax annotation-configs delete <annotation-config-id> [--force]

Supported annotation config types:

Type	Required options	Optional options
freeform	(none)	—
continuous	--min-score, --max-score	--optimization-direction
categorical	--value (repeat for multiple labels, e.g. --value good --value bad)	--optimization-direction

Cache

Manage the local cache. The CLI caches downloaded resource data (e.g., dataset examples) locally as Parquet files to avoid redundant API calls. When you fetch a dataset's examples, the results are stored on disk so subsequent requests for the same version load instantly. The cache is automatically invalidated when a resource's updated_at timestamp changes, so you always get fresh data when something changes on the server.

Caching is enabled by default and can be toggled in your profile configuration:

[storage]
cache_enabled = true

# Clear the cache
ax cache clear

Global Options

Available for all commands:

• --profile, -p <name>: Use a specific configuration profile
• --output, -o <format>: Set output format (table, json, csv, parquet, or a file path)
• --help, -h: Show help message

Note: --verbose, -v is available on each individual subcommand (e.g., ax datasets list --verbose) rather than as a top-level flag.

Usage Examples

Creating a Dataset from a CSV File

ax datasets create \
  --name "Customer Churn Dataset" \
  --space-id sp_abc123 \
  --file ./data/churn.csv

Exporting Dataset List to JSON

ax datasets list --space-id sp_abc123 --output json > datasets.json

Exporting Dataset Examples

# Export to a timestamped directory
ax datasets export ds_xyz789

# Export a specific version
ax datasets export ds_xyz789 --version-id ver_abc123

# Pipe to jq for processing
ax datasets export ds_xyz789 --stdout | jq '.[].input'

Exporting Experiment Runs

# Export all runs to a timestamped directory
ax experiments export exp_abc123

# Pipe to stdout for processing
ax experiments export exp_abc123 --stdout | jq '.[] | select(.output != null)'

Exporting Spans by Trace ID

# Export all spans in a trace
ax spans export proj_abc123 --trace-id tr_xyz789

# Export a session's spans to stdout
ax spans export proj_abc123 --session-id sess_456 --stdout

# Export with a custom lookback window
ax spans export proj_abc123 --trace-id tr_xyz789 --days 7

Using a Different Profile for a Command

ax datasets list --space-id sp_abc123 --profile production

Exporting Spans

# Export all spans from a project
ax spans export proj_abc123

# Export error spans
ax spans export proj_abc123 --filter "status_code = 'ERROR'" --limit 100

# Export spans in a time window to stdout
ax spans export proj_abc123 --start-time 2024-01-01T00:00:00Z --end-time 2024-01-02T00:00:00Z --stdout

Listing Traces and Exporting to Parquet

# List root traces in a project
ax traces list proj_abc123

# Export slow traces to Parquet for analysis
ax traces list proj_abc123 --filter "latency_ms > 2000" --limit 500 --output traces_slow.parquet

# List traces in JSON format
ax traces list proj_abc123 --output json

Pagination

List more datasets using pagination:

# First page
ax datasets list --space-id sp_abc123 --limit 20

# Next page (use cursor from previous response)
ax datasets list --space-id sp_abc123 --limit 20 --cursor <cursor-value>

Working with Multiple Environments

# Setup profiles for different environments
ax profiles create  # Create "production" profile
ax profiles create  # Create "staging" profile

# Switch contexts
ax profiles use production
ax datasets list --space-id sp_prod123

ax profiles use staging
ax datasets list --space-id sp_stage456

Filtering Spans by Status

ax spans export <project> --filter "status_code = 'ERROR'" --stdout

Listing Traces in a Time Window

ax traces list <project-id> \
  --start-time 2024-01-01T00:00:00Z \
  --end-time 2024-01-02T00:00:00Z

Advanced Topics

Output Formats

The CLI supports multiple output formats:

1. Table (default): Human-readable table format
2. JSON: Machine-readable JSON
3. CSV: Comma-separated values
4. Parquet: Apache Parquet columnar format

Set default format in profiles:

ax profiles create  # Select output format during setup

Or override per command:

ax datasets list --output json
ax datasets list --output datasets.csv
ax datasets list --output datasets.parquet

Programmatic Usage

Integrate with scripts:

#!/bin/bash

# Export datasets to JSON
DATASETS=$(ax datasets list --space-id sp_abc123 --output json)

# Process with jq
echo "$DATASETS" | jq '.data[] | select(.name | contains("test"))'

# Export to file
ax datasets export ds_xyz789

Environment Variables

The CLI respects these environment variables:

• ARIZE_API_KEY: Your Arize API key
• ARIZE_REGION: Region (US, EU, etc.)
• Any other ARIZE_* variables will be detected during ax profiles create

Debugging

Enable verbose mode to see detailed SDK logs:

ax datasets list --space-id sp_abc123 --verbose

Troubleshooting

Configuration Issues

Problem: Profile 'default' not found.

Solution: Run ax profiles create to create a configuration profile.

---

Problem: Invalid API key

Solution: Verify your API key:

1. Check your configuration: ax profiles show
2. Refresh your API key from the Arize UI
3. Update your profiles: ax profiles create (overwrite existing)

---

Connection Issues

Problem: Connection refused or SSL errors

Solution:

1. Check your routing configuration: ax profiles show
2. Verify network connectivity
3. For on-premise installations, ensure single_host is configured correctly
4. For SSL issues, check security.request_verify setting (use with caution)

---

Shell Completion Not Working

Problem: Tab completion doesn't work

Solution:

1. Verify completion is installed: Run the installation command for your shell
2. Reload your shell or open a new terminal
3. Ensure ax is in your PATH: which ax

---

Getting Help

Command-specific Help

Every command has detailed help:

ax --help
ax datasets --help
ax datasets create --help
ax profiles --help

Support

• Documentation: https://arize.com/docs/api-clients/cli/
• Bug Reports: GitHub Issues
• Community: Arize Community Slack
• Email: [e-mail]

Contributing

We welcome contributions!

• For developers: See DEVELOPMENT.md for architecture, code structure, and development guide
• For contributors: See CONTRIBUTING.md for contribution guidelines (coming soon)

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Changelog

See CHANGELOG.md for release notes and version history.

---

Built with ❤️ by Arize AI