cmdop-llm

cmdop-llm is a framework-neutral Python package for multi-provider LLM calls. It includes chat completions, structured extraction and repair, embeddings, vision, image generation/editing, translation, model routing, cost tracking, and an optional Pydantic AI agent harness.

The package does not depend on Django or django-cfg. Pass credentials directly to clients or configure them with environment variables.

Installation

pip install cmdop-llm

This is a library: it installs no command. pip install inside your project's virtualenv is the correct and only instruction for it — there is nothing to put on your PATH.

The agent plane (cmdop_llm.agent, a Pydantic AI harness) installs with it since 0.1.16. Importing cmdop_llm alone still loads no Pydantic AI; the [agent] extra remains as an empty alias for older pins.

Python 3.12 or newer is required.

Configuration

Explicit credentials take precedence:

from cmdop_llm import LLMClient

client = LLMClient(apikey_openrouter="your-key")
response = client.chat_completion(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.content)

When a credential is omitted, cmdop-llm reads these environment variables:

• OPENROUTER_API_KEY or CMDOP_LLM_KEYS__OPENROUTER
• OPENAI_API_KEY or CMDOP_LLM_KEYS__OPENAI

A host application can inject a configuration object without introducing a framework dependency:

from cmdop_llm import LLMConfig, set_config

set_config(
    LLMConfig(
        project_name="my-service",
        api_keys={"openrouter": "your-key"},
    )
)

Use reset_config() in tests after changing process-wide configuration.

Common operations

from pydantic import BaseModel

from cmdop_llm import LLMRouter, embed_fast, extract_many


class Product(BaseModel):
    name: str
    price: float


router = LLMRouter()
product = router.parse(
    schema=Product,
    text="The keyboard costs 99.50 USD",
)

vector = embed_fast("semantic search text")
products = extract_many(Product, ["Mouse: $20", "Display: $300"])

Other public entry points include LLMClient, VisionClient, ImageGenClient, ImageEditClient, DjangoTranslator (the historical name; it has no Django dependency), and structured-output repair utilities.

Model catalog

Rather than pinning a model id, ask for a role and get the curated chain:

from cmdop_llm.catalog import ModelRole, provider_for, recommend, traits

recommend(ModelRole.CLASSIFY)          # ordered chain, best-first
traits("openai/gpt-4o-mini").verdict(ModelRole.EXTRACTION)
provider_for("moonshotai/kimi-k2.6")   # which provider serves this slug

The catalog is hand-curated from measured probes, not scraped: each entry carries per-role verdicts and known operational pitfalls, and the ordering of each chain is a deliberate decision recorded alongside it. recommend() returns an ordered list so a caller can fall through on failure; the first entry is the intended default.

Entries live in catalog/models/, one module per serving provider, because what differs per entry is the serving stack — who bills it, what the price basis is, which flags apply.

Transport and agent planes

The package root is the transport plane and intentionally does not import the heavier agent dependencies:

from cmdop_llm import LLMClient, embed_fast

The harness is an explicit submodule:

from cmdop_llm.agent.runtime.runner import run_agent_sync

This keeps ordinary transport imports lightweight. See @docs/agent/README.md for the harness architecture and host protocols.

Notifications and cache

Monitoring uses an in-process TTL cache by default. Email and Telegram are not hard-coded. A host can register a notification callback:

from cmdop_llm import set_notification_handler


def notify(subject: str, message: str, context: dict) -> dict[str, bool]:
    # Send through the host's email, Telegram, queue, etc.
    return {"email": True, "telegram": False}


set_notification_handler(notify)

Development and release

python -m pip install -e ".[dev]"
pytest
python ../publishing/publish.py cmdop-llm --build-only
python ../publishing/publish.py cmdop-llm --test

The long-form architecture, provider notes, research, and operational guidance are copied under @docs/.

Temporary provider media

Publish an image or MP4 for a provider that requires a public URL without giving the application Cloudflare credentials:

from datetime import timedelta
from pathlib import Path

from cmdop_llm.media.publication import SdkRouterPublisher

publisher = SdkRouterPublisher()
media = publisher.publish(
    Path("frame.png"),
    filename="frame.png",
    content_type="image/png",
    expires_in=timedelta(hours=2),
)

provider_url = media.url.get_secret_value()
# Submit provider_url, then revoke on the provider job's terminal state.
publisher.revoke(media)

The returned URL and revoke token are masked in logs. Candidate identity, provider render windows and publication manifests remain the caller's domain; this package owns only exact-byte upload and revoke transport.

Automatic media transport

Provider adapters should accept a path, bytes, or Django FieldFile and pass it through the shared router. The destination policy is the single source of truth for whether that API prefers an inline data URL, multipart bytes, or a temporary public URL:

from pathlib import Path

from cmdop_llm import MediaTarget, MediaTransportRouter

router = MediaTransportRouter()

# OpenRouter accepts inline images, so this does not upload anything.
inline = router.prepare_for(MediaTarget.OPENROUTER_IMAGE, Path("frame.png"))
openrouter_value = inline.prepared.as_reference()

# URL-only providers transparently use https://cdn.sdkrouter.com.
lease = router.prepare_for(MediaTarget.PUBLIC_URL, Path("frame.png"))
provider_url = lease.prepared.as_reference()

# Keep the lease while an asynchronous provider may still fetch the URL.
# Revoke it on the remote job's terminal state; TTL remains the safety net.
lease.revoke()

normalize_image_input() already uses this router for local OpenRouter images. New OCR, Image-to-Video, and other provider adapters must declare a MediaTransportPolicy instead of implementing their own base64/CDN branching. Existing public HTTPS URLs pass through without being copied and are marked verified=False, because the package deliberately performs no surprising network download; their size and bytes remain the destination's preflight responsibility. Local inputs are bounded, structurally checked, hashed, and represented using the first feasible transport in the destination policy. Inline data and signed URLs are excluded from normal model repr/dumps.