space-evals-client-anthropic

Anthropic client plugin for space-evals. Lets you use Anthropic models (Claude Sonnet, Claude Opus, Claude Haiku, etc.) as the target, customer, or judge in your space-evals test suite.

Installation

pip install space-evals-client-anthropic

This installs the core space-evals package and the anthropic SDK as dependencies.

Setup

Set your API key:

export ANTHROPIC_API_KEY=sk-ant-...

Or use a custom env var name in your config (see below).

Usage

Via config file

Add provider: anthropic to any client role in your space-evals.yaml:

clients:
  target:
    provider: anthropic
    model: claude-sonnet-4-20250514

  customer:
    provider: anthropic
    model: claude-haiku-4-5-20251001

  judge:
    provider: anthropic
    model: claude-sonnet-4-20250514

Then run your tests:

space-evals run ./tests/

Via programmatic API

import asyncio
from space_evals.engine import run_tests
from space_evals.clients.base import get_client

target = get_client("anthropic", model="claude-sonnet-4-20250514")

result = asyncio.run(run_tests(
    path="./tests/",
    target_client=target,
))

Configuration

Example with extra params

clients:
  target:
    provider: anthropic
    model: claude-sonnet-4-20250514
    api_key_env: MY_ANTHROPIC_KEY
    params:
      temperature: 0
      max_tokens: 4096

What this plugin does

• Translates space-evals's Message format to Anthropic's messages API
• Extracts tool use blocks from responses into space-evals's ToolCall format
• Handles system prompts via Anthropic's system parameter (used by the customer and judge roles)
• Lazy-loads the Anthropic SDK on first use
• Defaults max_tokens to 1024 if not specified

Requirements

• Python >= 3.11
• space-evals >= 0.1.0
• anthropic >= 0.30
• ANTHROPIC_API_KEY environment variable (or custom via api_key_env)

License

MIT