space-evals-client-openai

OpenAI client plugin for space-evals. Lets you use OpenAI models (GPT-4o, GPT-4, etc.) as the target, customer, or judge in your space-evals test suite.

Installation

pip install space-evals-client-openai

This installs the core space-evals package and the openai SDK as dependencies.

Setup

Set your API key:

export OPENAI_API_KEY=sk-...

Or use a custom env var name in your config (see below).

Usage

Via config file

Add provider: openai to any client role in your space-evals.yaml:

clients:
  target:
    provider: openai
    model: gpt-4o

  customer:
    provider: openai
    model: gpt-4o-mini

  judge:
    provider: openai
    model: gpt-4o

Then run your tests:

space-evals run ./tests/

Via programmatic API

import asyncio
from space_evals.engine import run_tests
from space_evals.clients.base import get_client

target = get_client("openai", model="gpt-4o")

result = asyncio.run(run_tests(
    path="./tests/",
    target_client=target,
))

Configuration

Example with extra params

clients:
  target:
    provider: openai
    model: gpt-4o
    api_key_env: MY_OPENAI_KEY
    params:
      temperature: 0
      max_tokens: 2048

What this plugin does

• Translates space-evals's Message format to OpenAI's chat completions API
• Extracts tool calls from responses into space-evals's ToolCall format
• Handles system prompts (used by the customer and judge roles)
• Lazy-loads the OpenAI SDK on first use

Requirements

• Python >= 3.11
• space-evals >= 0.1.0
• openai >= 1.0
• OPENAI_API_KEY environment variable (or custom via api_key_env)

License

MIT