cmdop-llm 0.1.2

Python SDK for CMDOP LLM Service - OpenAI-compatible API for 200+ AI models

CMDOP LLM Python SDK

Python SDK for CMDOP LLM Service - OpenAI-compatible API for 200+ AI models.

Installation

pip install cmdop-llm

Quick Start

from cmdop_llm import CmdopLLM

client = CmdopLLM(api_key="your-api-key")

response = client.chat.completions.create(
    model="anthropic/claude-3.5-sonnet",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Features

• Drop-in OpenAI replacement - Same API, different models
• 200+ Models - GPT-4, Claude, Llama, Mistral, Gemini via single endpoint
• Streaming - Real-time token streaming
• Tool Calling - Function calling support
• Structured Output - Parse responses to Pydantic models
• Embeddings - Text embeddings generation
• Vision & OCR - Image analysis and text extraction
• Image Generation - FLUX, DALL-E and other models
• Async Support - Full async/await support

Environment Variables

export CMDOP_API_KEY="your-api-key"
export CMDOP_BASE_URL="https://llm.cmdop.com/v1"  # Optional, default

Usage Examples

Chat Completion

from cmdop_llm import CmdopLLM

client = CmdopLLM()

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing."}
    ],
    temperature=0.7,
    max_tokens=1000,
)
print(response.choices[0].message.content)

Streaming

stream = client.chat.completions.create(
    model="anthropic/claude-3.5-sonnet",
    messages=[{"role": "user", "content": "Write a poem."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Tool Calling

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"}
            },
            "required": ["location"]
        }
    }
}]

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
    tool_choice="auto",
)

if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    print(f"Function: {tool_call.function.name}")
    print(f"Arguments: {tool_call.function.arguments}")

Vision Analysis

result = client.vision.analyze(
    image_url="https://example.com/image.jpg",
    prompt="Describe this image"
)
print(result.description)
print(result.extracted_text)

OCR Text Extraction

result = client.ocr.extract(
    image_url="https://example.com/document.png"
)
print(result.text)

Image Generation

response = client.images.generate(
    model="black-forest-labs/FLUX.1-schnell",
    prompt="A futuristic cityscape",
    size="1024x1024",
)
print(response.data[0].url)

Embeddings

response = client.embeddings.create(
    model="openai/text-embedding-3-small",
    input="Hello, world!"
)
print(response.data[0].embedding[:5])  # First 5 dimensions

Structured Output with Pydantic

from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int
    city: str

# Parse response directly into Pydantic model
response = client.beta.chat.completions.parse(
    model="openai/gpt-4o",
    messages=[
        {"role": "user", "content": "Extract: John is 30 years old and lives in Tokyo"}
    ],
    response_format=Person,
)

person = response.choices[0].message.parsed
print(f"{person.name}, {person.age}, {person.city}")  # John, 30, Tokyo

JSON Schema Response Format

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "List 3 colors"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "colors",
            "schema": {
                "type": "object",
                "properties": {
                    "colors": {"type": "array", "items": {"type": "string"}}
                },
                "required": ["colors"]
            }
        }
    }
)

Async Usage

import asyncio
from cmdop_llm import AsyncCmdopLLM

async def main():
    client = AsyncCmdopLLM()

    response = await client.chat.completions.create(
        model="openai/gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    print(response.choices[0].message.content)

asyncio.run(main())

Available Models

Access 200+ models including:

• OpenAI: gpt-4o, gpt-4o-mini, gpt-4-turbo
• Anthropic: claude-3.5-sonnet, claude-3-opus, claude-3-haiku
• Google: gemini-pro, gemini-1.5-pro
• Meta: llama-3.1-405b, llama-3.1-70b
• Mistral: mistral-large, mixtral-8x22b
• Image: FLUX.1-schnell, FLUX.1-pro, stable-diffusion-xl

Use model format: provider/model-name (e.g., openai/gpt-4o)

API Reference

CmdopLLM

CmdopLLM(
    api_key: str = None,       # From CMDOP_API_KEY env if not set
    base_url: str = None,      # Default: https://llm.cmdop.com
    timeout: float = None,     # Request timeout
    max_retries: int = 2,      # Retry count
)

Resources

• client.chat.completions - Chat completions (OpenAI compatible)
• client.beta.chat.completions.parse() - Structured output with Pydantic
• client.embeddings - Text embeddings (OpenAI compatible)
• client.images - Image generation (OpenAI compatible)
• client.models - List available models
• client.vision - Vision analysis (CMDOP specific)
• client.ocr - OCR extraction (CMDOP specific)

License

MIT