litellm-td-llm-provider 0.1.1

LiteLLM provider for Treasure Data LLM Proxy

LiteLLM TD LLM Proxy Provider

Custom LiteLLM provider for Treasure Data's LLM Proxy API.

Installation

Using uv (recommended)

cd litellm-td-llm-provider
uv sync
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

Using pip

cd litellm-td-llm-provider
pip install -e .

Usage

Basic Completion

import litellm
from litellm_td_llm_provider import register_td_provider

# Register the TD provider
register_td_provider()

# Use with LiteLLM
response = litellm.completion(
    model="td/claude-sonnet-4-5",
    messages=[{"role": "user", "content": "Hello!"}],
    api_key="your-td-api-key",
    api_base="https://llm-proxy.us01.treasuredata.com"
)

Async Completion

import asyncio
import litellm
from litellm_td_llm_provider import register_td_provider

register_td_provider()

async def main():
    response = await litellm.acompletion(
        model="td/claude-sonnet-4-5",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    print(response.choices[0].message.content)

asyncio.run(main())

Tool Calling

import litellm
from litellm_td_llm_provider import register_td_provider

register_td_provider()

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"},
                },
                "required": ["location"],
            },
        },
    }
]

response = litellm.completion(
    model="td/claude-sonnet-4-5",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
)

if response.choices[0].message.tool_calls:
    for tool_call in response.choices[0].message.tool_calls:
        print(f"Tool: {tool_call.function.name}")
        print(f"Arguments: {tool_call.function.arguments}")

Configuration

Available Models

The following Claude 4.5 models are available through TD LLM Proxy:

Model	Model ID	Alias
Claude Sonnet 4.5	claude-sonnet-4-5-20250929	claude-sonnet-4-5
Claude Haiku 4.5	claude-haiku-4-5-20251001	claude-haiku-4-5
Claude Opus 4.5	claude-opus-4-5-20251101	claude-opus-4-5

Note:

• Aliases (e.g., claude-sonnet-4-5) automatically point to the latest version
• Use specific versions (e.g., claude-sonnet-4-5-20250929) if you need to pin to a particular model snapshot

This provider uses claude-sonnet-4-5 as the default model.

Site Endpoints

• us01: https://llm-proxy.us01.treasuredata.com
• jp01: https://llm-proxy.treasuredata.co.jp
• eu01: https://llm-proxy.eu01.treasuredata.com
• ap02: https://llm-proxy.ap02.treasuredata.com
• ap03: https://llm-proxy.ap03.treasuredata.com
• dev-us01: https://llm-proxy-development.us01.treasuredata.com
• dev-eu01: https://llm-proxy-development.eu01.treasuredata.com
• stg-us01: https://llm-proxy-staging.us01.treasuredata.com
• stg-jp01: https://llm-proxy-staging.treasuredata.co.jp
• stg-ap03: https://llm-proxy-staging.ap03.treasuredata.com

Environment Variables

export TD_LLM_API_KEY="your-api-key"
export TD_LLM_SITE="us01"  # Optional, defaults to us01

Features

• ✅ Streaming support
• ✅ Non-streaming completion
• ✅ Async completion (acompletion)
• ✅ Tool calling support (OpenAI → Anthropic format conversion)
• ✅ Multi-site support
• ✅ Anthropic Messages API compatible
• ✅ OpenAI-compatible interface

Development

Setup

# Install dependencies (creates venv automatically)
uv sync

# Activate virtual environment
source .venv/bin/activate

# Run tests
uv run pytest

Using Makefile

make install-dev  # Install dev dependencies
make format       # Format code with ruff
make lint         # Run linting
make check        # Run type checking with pyright
make test         # Run tests
make test-cov     # Run tests with coverage
make check-all    # Run all checks (format, lint, check, test)
make clean        # Clean cache and build artifacts

Release Process

1. Update version in pyproject.toml
2. Create and push a git tag:

   git tag v0.1.0
   git push origin v0.1.0
   

3. GitHub Actions will automatically build and publish to PyPI

Required GitHub Secret:

• PYPI_TOKEN: PyPI API token for publishing

Running Examples

# Set credentials
export TD_LLM_API_KEY="your-api-key"
export TD_LLM_SITE="us01"

# Run basic example
python examples/basic_usage.py

# Run site selection example
python examples/site_selection.py

Implementation Details

This provider is based on the TypeScript implementation in tdx-studio/electron/services/llm-proxy-client.ts and provides:

1. Endpoint Management: Multi-region support with automatic site resolution
2. Streaming: Server-Sent Events (SSE) parsing for real-time responses
3. Error Handling: Proper error propagation and API error messages
4. LiteLLM Integration: Custom provider registration with standard LiteLLM interface

API Compatibility

The TD LLM Proxy API is Anthropic-compatible and supports:

• /v1/messages endpoint
• x-api-key authentication
• anthropic-version: 2023-06-01 header
• Streaming via SSE with stream: true