lunar-sdk 0.2.0

Python SDK for Lunar LLM Inference API - OpenAI-compatible with fallbacks

Lunar SDK

Python SDK for Lunar LLM Inference API - OpenAI-compatible with intelligent fallbacks.

Installation

pip install lunar

Quick Start

from lunar import Lunar

# Initialize client (uses LUNAR_API_KEY env var)
client = Lunar()

# Chat completion
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
print(f"Cost: ${response.usage.total_cost_usd}")

Authentication

Set your API key via environment variable:

export LUNAR_API_KEY="your-api-key"

Or pass it directly:

client = Lunar(api_key="your-api-key")

Features

Chat Completions

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

print(response.choices[0].message.content)

Streaming

Stream responses token by token:

# Synchronous streaming
stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Write a short story"}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Async streaming:

from lunar import AsyncLunar

async with AsyncLunar() as client:
    stream = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Write a short story"}],
        stream=True
    )

    async for chunk in stream:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="", flush=True)

Text Completions

response = client.completions.create(
    model="gpt-4o-mini",
    prompt="Once upon a time",
    max_tokens=100
)

print(response.choices[0].text)

Fallbacks

Lunar automatically falls back to alternative models when the primary model fails with infrastructure errors (5xx, rate limits, timeouts).

# Per-request fallbacks
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    fallbacks=["claude-3-haiku", "llama-3.1-8b"]
)

# Global fallbacks via config
client = Lunar(
    fallbacks={
        "gpt-4o-mini": ["claude-3-haiku", "llama-3.1-8b"],
        "gpt-4": ["claude-3-opus"]
    }
)

Fallback behavior:

• Triggers fallback: 5xx errors, 429 rate limit, connection errors, timeouts
• Does NOT trigger fallback: 400 bad request, 401 auth error, 403 forbidden (these are client errors that won't be fixed by another model)

Force Provider

# Force a specific provider
response = client.chat.completions.create(
    model="openai/gpt-4o-mini",  # Forces OpenAI provider
    messages=[{"role": "user", "content": "Hello!"}]
)

List Models and Providers

# List available models
models = client.models.list()
for model in models:
    print(f"{model.id} (owned by {model.owned_by})")

# List providers for a model
providers = client.providers.list(model="gpt-4o-mini")
for provider in providers:
    print(f"{provider.id}: {provider.type} (enabled: {provider.enabled})")

Cost Tracking

Every response includes detailed cost information:

response = client.chat.completions.create(...)

print(f"Input tokens: {response.usage.prompt_tokens}")
print(f"Output tokens: {response.usage.completion_tokens}")
print(f"Input cost: ${response.usage.input_cost_usd}")
print(f"Output cost: ${response.usage.output_cost_usd}")
print(f"Total cost: ${response.usage.total_cost_usd}")
print(f"Latency: {response.usage.latency_ms}ms")

Async Usage

from lunar import AsyncLunar

async with AsyncLunar() as client:
    response = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    print(response.choices[0].message.content)

Configuration

client = Lunar(
    api_key="your-api-key",           # Or use LUNAR_API_KEY env var
    base_url="https://api.lunar-sys.com", # Custom API endpoint
    timeout=60.0,                      # Request timeout in seconds
    num_retries=3,                     # Retries for transient errors
    max_connections=100,               # Max concurrent connections
    fallbacks={                        # Global fallback configuration
        "gpt-4o-mini": ["claude-3-haiku"]
    }
)

Error Handling

from lunar import (
    Lunar,
    LunarError,
    APIError,
    BadRequestError,
    AuthenticationError,
    RateLimitError,
    ServerError,
)

client = Lunar()

try:
    response = client.chat.completions.create(...)
except BadRequestError as e:
    print(f"Invalid request: {e}")
except AuthenticationError as e:
    print(f"Auth failed: {e}")
except RateLimitError as e:
    print(f"Rate limited: {e}")
except ServerError as e:
    print(f"Server error: {e}")
except APIError as e:
    print(f"API error [{e.status_code}]: {e}")
except LunarError as e:
    print(f"Lunar error: {e}")

License

MIT