arize_toolkit 1.0.11

📚 Documentation

Quick Links

• Home - Main documentation page
• Quickstart Guide - Get started quickly with Arize Toolkit
• CLI Reference - Command-line interface documentation

Tools Documentation

Extensions

For Developers

• Development Guide - Information about extending the toolkit
• Integration Tests - Running integration tests
• Claude Code Skills - AI-assisted development workflows

Claude Code Skills

This repository includes Claude Code skills for AI-assisted development:

Use /arize-graphql-analytics or /new-query-workflow in Claude Code to invoke these skills.

Disclaimer

Although this package is used for development work with and within the Arize platform, it is not an Arize product. It is a open source project developed and maintained by an Arize Engineer. Feel free to add issues or reach out for help in the Arize community Slack channel.

Overview

Arize Toolkit is a set of tools packaged as a Python client that lets you easily interact with Arize AI APIs. Here's a quick overview of the main features in the current release:

• Access and manage models
• Retrieve performance metrics over a time period
• Retrieve inference volume over a time period
• Create, copy, and manage custom metrics
• Create, copy, and manage monitors and alerting
• Work with LLM features like prompts and annotations
• Import data from cloud storage (S3, GCS, Azure) and databases (BigQuery, Snowflake, Databricks)
• Create, update, and delete data import jobs with full lifecycle management
• Prompt Optimization Extension (optional): Automatically optimize prompts using meta-prompt techniques with feedback from evaluators
• Command-Line Interface (optional): Manage all Arize resources directly from the terminal with Rich table output and JSON mode

Installation

pip install arize_toolkit

Optional Dependencies

Command-Line Interface

To use the arize_toolkit CLI, install with the cli extras:

pip install arize_toolkit[cli]

Prompt Optimization Extension

For automated prompt optimization using meta-prompt techniques, install with the prompt_optimizer extras:

pip install arize_toolkit[prompt_optimizer]

Command-Line Interface

The CLI wraps all Client functionality so you can manage models, monitors, prompts, and more directly from the terminal.

Quick Start

# One-time setup — saves credentials to ~/.arize_toolkit/config.toml
arize_toolkit config init

# List models (Rich table output by default)
arize_toolkit models list

# Same thing, using the "projects" alias
arize_toolkit projects list

# JSON output, pipe to jq
arize_toolkit --json models list | jq '.[].name'

# Get monitor details for a specific model
arize_toolkit monitors get "accuracy-alert" --model "fraud-detection-v3"

# Create a performance monitor
arize_toolkit monitors create-performance "accuracy-alert" \
    --model "fraud-detection-v3" \
    --environment production \
    --performance-metric accuracy \
    --threshold 0.95

# Use a named profile for a different environment
arize_toolkit --profile staging spaces list

Global Options

Command Groups

See the full CLI documentation for detailed usage and examples.

Client Setup

The Client class is the entrypoint for interacting with the toolkit. It provides maintains the connection information for making requests to the Arize APIs, and offers a wide range of operations for interacting with models, monitors, dashboards, and more.

API Key

To create a client, you need to provide your Arize API key. Use this reference to get your API key from the Arize UI.

Organization and Space

You will also need to provide an organization name and space name. To give some context, models are scoped to a space, and the space is scoped to an organization. These can be found by navigating to the Arize UI and looking at the upper left dropdown from the landing page. You will first select an organization from the dropdown and then pick the space from the available spaces in that organization.

For the example below, the organization is Demo Org and the space is Test Space.

For On Prem deployments

For SaaS users, the default API endpoint is always going to be https://api.arize.com. If you are using an on prem deployment of Arize, you will need to provide the api_url parameter. This parameters should just be the base url of your Arize instance.