cmdop-llm 0.1.6

Python SDK for CMDOP LLM Service - OpenAI-compatible API for 200+ AI models

CMDOP LLM Python SDK

Python SDK for CMDOP LLM Service - OpenAI-compatible API for 200+ AI models.

Installation

pip install cmdop-llm

Quick Start

from cmdop_llm import CmdopLLM

client = CmdopLLM(api_key="your-api-key")

response = client.chat.completions.create(
    model="anthropic/claude-3.5-sonnet",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Features

• Drop-in OpenAI replacement - Same API, different models
• 200+ Models - GPT-4, Claude, Llama, Mistral, Gemini via single endpoint
• Streaming - Real-time token streaming
• Tool Calling - Function calling support
• Structured Output - Parse responses to Pydantic models
• Embeddings - Text embeddings generation
• Vision & OCR - Image analysis and text extraction
• Image Generation - FLUX, DALL-E and other models
• Web Search - AI-powered web search with citations
• Async Support - Full async/await support

Environment Variables

export CMDOP_API_KEY="your-api-key"
export CMDOP_BASE_URL="https://llm.cmdop.com/v1"  # Optional, default

Usage Examples

Chat Completion

from cmdop_llm import CmdopLLM

client = CmdopLLM()

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing."}
    ],
    temperature=0.7,
    max_tokens=1000,
)
print(response.choices[0].message.content)

Streaming

stream = client.chat.completions.create(
    model="anthropic/claude-3.5-sonnet",
    messages=[{"role": "user", "content": "Write a poem."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Tool Calling

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"}
            },
            "required": ["location"]
        }
    }
}]

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
    tool_choice="auto",
)

if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    print(f"Function: {tool_call.function.name}")
    print(f"Arguments: {tool_call.function.arguments}")

Web Search

from cmdop_llm import CmdopLLM, UserLocation

client = CmdopLLM()

# Basic web search
result = client.search.web("What is the capital of France?")
print(result.content)

# Print citations
for citation in result.citations:
    print(f"- {citation.title}: {citation.url}")

# Search with options
result = client.search.web(
    "Latest AI news",
    max_searches=5,
    allowed_domains=["bbc.com", "cnn.com", "reuters.com"],
    user_location=UserLocation(country="US", city="New York"),
)
print(result.content)

URL Fetch & Analysis

# Fetch and analyze a specific URL
result = client.search.fetch(
    url="https://en.wikipedia.org/wiki/Python_(programming_language)",
    prompt="What are the key features of Python? List top 5.",
)
print(result.content)

Vision Analysis

result = client.vision.analyze(
    image_url="https://example.com/image.jpg",
    prompt="Describe this image"
)
print(result.description)
print(result.extracted_text)

OCR Text Extraction

result = client.ocr.extract(
    image_url="https://example.com/document.png"
)
print(result.text)

Image Generation

response = client.images.generate(
    model="black-forest-labs/flux.2-pro",
    prompt="A futuristic cityscape",
    size="1024x1024",
)
print(response.data[0].url)

Embeddings

response = client.embeddings.create(
    model="openai/text-embedding-3-small",
    input="Hello, world!"
)
print(response.data[0].embedding[:5])  # First 5 dimensions

Structured Output with Pydantic

from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int
    city: str

# Parse response directly into Pydantic model
response = client.beta.chat.completions.parse(
    model="openai/gpt-4o",
    messages=[
        {"role": "user", "content": "Extract: John is 30 years old and lives in Tokyo"}
    ],
    response_format=Person,
)

person = response.choices[0].message.parsed
print(f"{person.name}, {person.age}, {person.city}")  # John, 30, Tokyo

JSON Schema Response Format

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "List 3 colors"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "colors",
            "schema": {
                "type": "object",
                "properties": {
                    "colors": {"type": "array", "items": {"type": "string"}}
                },
                "required": ["colors"]
            }
        }
    }
)

Async Usage

import asyncio
from cmdop_llm import AsyncCmdopLLM

async def main():
    client = AsyncCmdopLLM()

    # Chat
    response = await client.chat.completions.create(
        model="openai/gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    print(response.choices[0].message.content)

    # Web Search
    result = await client.search.web("Latest tech news")
    print(result.content)

asyncio.run(main())

Available Models

Access 200+ models including:

• OpenAI: gpt-4o, gpt-4o-mini, gpt-4-turbo
• Anthropic: claude-3.5-sonnet, claude-3-opus, claude-3-haiku
• Google: gemini-pro, gemini-1.5-pro
• Meta: llama-3.1-405b, llama-3.1-70b
• Mistral: mistral-large, mixtral-8x22b
• Image: flux.2-pro, flux.2-flex, gemini-2.5-flash-image

Use model format: provider/model-name (e.g., openai/gpt-4o)

API Reference

CmdopLLM

CmdopLLM(
    api_key: str = None,       # From CMDOP_API_KEY env if not set
    base_url: str = None,      # Default: https://llm.cmdop.com/v1
    timeout: float = None,     # Request timeout
    max_retries: int = 2,      # Retry count
)

Resources

Resource	Description
client.chat.completions	Chat completions (OpenAI compatible)
client.beta.chat.completions.parse()	Structured output with Pydantic
client.embeddings	Text embeddings (OpenAI compatible)
client.images	Image generation (OpenAI compatible)
client.models	List available models
client.vision	Vision analysis (CMDOP specific)
client.ocr	OCR extraction (CMDOP specific)
client.search	Web search and URL fetch (CMDOP specific)

Search Methods

# Web search with AI-summarized results
client.search.web(
    query: str,                      # Search query
    model: str = "claude-3-5-haiku-20241022",
    max_tokens: int = 1024,
    max_searches: int = 5,           # Max web searches (1-10)
    allowed_domains: list[str] = None,
    blocked_domains: list[str] = None,
    user_location: UserLocation = None,
) -> WebSearchResponse

# Fetch and analyze URL content
client.search.fetch(
    url: str,                        # URL to fetch
    prompt: str = "Summarize this page",
    model: str = "claude-3-5-haiku-20241022",
    max_tokens: int = 1024,
) -> WebSearchResponse

Response Types

# WebSearchResponse
response.id          # Unique response ID
response.content     # AI-generated response
response.citations   # List of SearchCitation
response.model       # Model used
response.usage       # SearchUsage (input_tokens, output_tokens)
response.stop_reason # Why response stopped

# SearchCitation
citation.title       # Source page title
citation.url         # Source URL
citation.cited_text  # Quoted text (optional)

# UserLocation
UserLocation(
    country="US",    # ISO 3166-1 alpha-2 code
    city="New York",
    region="NY",
)

License

MIT