aidial-adapter-anthropic 0.14.0

Package implementing adapter from DIAL Chat Completions API to Anthropic API

Python SDK for adapter from DIAL API to Anthropic API

• Overview
• Anthropic API passthrough
  • Usage
  • Proxied endpoints
  • Supported backends
• Prompt caching
  • Automatic caching
  • Explicit cache breakpoints
  • TTL support
• Development Environment
  • Setup
  • Lint
  • Test
  • Clean
  • Build
  • Publish
  • Git hooks

---

Overview

The framework provides adapter from AI DIAL Chat Completion API to Anthropic Messages API.

---

Anthropic API passthrough

In addition to the DIAL-to-Anthropic adapter, the library exposes a transparent passthrough for the native Anthropic Messages API.

The exposed Anthropic Messages API is compatible with the vanilla Anthropic Client from Anthropic SDK:

from anthropic import Anthropic, AsyncAnthropic
client = Anthropic(api_key="...", base_url="${ADAPTER_ORIGIN}/anthropic")

The upstream errors are relayed to the caller in the native Anthropic error schema.

Usage

Mount the passthrough onto any Starlette/FastAPI host application (e.g. a DIALApp) with mount_anthropic_api. The upstream client is chosen per request by a factory you supply:

from aidial_sdk import DIALApp
from anthropic import AsyncAnthropic
from aidial_adapter_anthropic.passthrough import mount_anthropic_api

app = DIALApp(...)

async def get_client(request):
    return AsyncAnthropic(api_key=...)

mount_anthropic_api(app, get_client)

The passthrough is mounted at /anthropic by default; pass path=... to change it. The get_client argument may also be a plain client instance instead of a factory.

Proxied endpoints

The following Anthropic endpoints are forwarded (relative to the mount path):

• POST /v1/messages — create a message (streaming and non-streaming)
• POST /v1/messages/batches — create a message batch
• POST /v1/messages/count_tokens — count tokens

Supported backends

The client factory may return any of the Anthropic SDK's async clients: AsyncAnthropic, AsyncAnthropicBedrock, AsyncAnthropicBedrockMantle, AsyncAnthropicVertex, and AsyncAnthropicFoundry.

The Bedrock backends require botocore, which is an optional dependency:

pip install aidial-adapter-anthropic[bedrock]

Endpoints a backend does not implement (e.g. Bedrock has no token-counting or batches route) surface as a 404 error.

---

Prompt caching

Automatic caching

Automatic caching is the simplest way to use prompt caching. A single top-level cache breakpoint instructs Anthropic to automatically apply a cache point to the last cacheable block of the request. This is ideal for multi-turn conversations where the growing message history should be cached automatically. See Automatic caching in the Anthropic docs.

To enable automatic caching, set custom_fields.cache_breakpoint at the top level of the Chat Completion request:

Top-level cache breakpoint

{
  "model": "claude-3-5-sonnet-20241022",
  "messages": [
    {"role": "user", "content": "Hello!"}
  ],
  "custom_fields": {
    "cache_breakpoint": {}
  }
}

Explicit cache breakpoints

Explicit cache breakpoints give fine-grained control over which parts of the prompt get cached. You can place a cache breakpoint on individual system messages, user/assistant messages, or tool definitions. See Explicit cache breakpoints in the Anthropic docs.

To add a breakpoint, set custom_fields.cache_breakpoint on a message or tool object:

System cache breakpoint

{
  "model": "claude-3-5-sonnet-20241022",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant with extensive knowledge.",
      "custom_fields": {
        "cache_breakpoint": {}
      }
    },
    {"role": "user", "content": "Hello!"}
  ]
}

Message cache breakpoint

{
  "model": "claude-3-5-sonnet-20241022",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {
      "role": "user",
      "content": "Here is a long document: ...",
      "custom_fields": {
        "cache_breakpoint": {}
      }
    },
    {"role": "user", "content": "Summarize it."}
  ]
}

Tools cache breakpoint

{
  "model": "claude-3-5-sonnet-20241022",
  "messages": [
    {"role": "user", "content": "What's the weather?"}
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {"type": "string"}
          },
          "required": ["location"]
        }
      },
      "custom_fields": {
        "cache_breakpoint": {}
      }
    }
  ]
}

TTL support

A cache breakpoint may include an optional ttl field. Supported values are 5m (5 minutes, default) and 1h (one hour). The ttl field is supported on both top-level and explicit breakpoints. See TTL support in the Anthropic docs.

Top-level cache breakpoint with TTL

{
  "model": "claude-3-5-sonnet-20241022",
  "messages": [
    {"role": "user", "content": "Hello!"}
  ],
  "custom_fields": {
    "cache_breakpoint": {
      "ttl": "1h"
    }
  }
}

---

Development Environment

This project requires Python ≥3.11 and Poetry ≥2.1.1 for dependency management.

Setup

1. Install Poetry. See the official installation guide.

2. (Optional) Specify custom Python or Poetry executables in .env.dev. This is useful if multiple versions are installed. By default, python and poetry are used.

   POETRY_PYTHON=path-to-python-exe
   POETRY=path-to-poetry-exe
   

3. Create and activate the virtual environment:

   make init_env
   source .venv/bin/activate
   

4. Install project dependencies (including linting, formatting, and test tools):

   make install
   

Lint

Run the linting before committing:

make lint

To auto-fix formatting issues run:

make format

Test

Run unit tests locally for available python versions:

make test

Run unit tests for the specific python version:

make test PYTHON=3.13

Clean

To remove the virtual environment and build artifacts run:

make clean

Build

To build the package run:

make build

Publish

To publish the package to PyPI run:

make publish

Git hooks

You may optionally install Git hooks that will automatically run the linting step on Git push. You only need to do it once for the given repository.

make install_git_hooks

[!IMPORTANT] This command doesn't work if you have already installed Git hooks locally or globally.