llm-zenmux

LLM plugin for ZenMux.ai — access every model available through ZenMux (OpenAI, Anthropic, Google, Moonshot, and more) from a single unified API key.

ZenMux exposes two API surfaces, both supported by this plugin:

• OpenAI Chat Completions (zenmux/openai/…, zenmux/google/…, zenmux/moonshotai/…, …)
• Anthropic Messages (zenmux/anthropic/claude-…)

The plugin auto-discovers the live model list from https://zenmux.ai/api/v1/models (cached for one hour) and registers every model under the zenmux/ namespace.

Installation

pip install llm-zenmux

Or with uv:

uv pip install llm-zenmux

API key setup

llm keys set zenmux

Paste your ZenMux API key when prompted. The key is stored in llm's encrypted key store and sent as the Authorization: Bearer … header (OpenAI surface) or x-api-key header (Anthropic surface).

Usage

List available models:

llm zenmux models

Prompt any model — the -m flag takes the full zenmux/<provider>/<model> ID:

llm -m zenmux/openai/gpt-4o "What is ZenMux?"
llm -m zenmux/anthropic/claude-sonnet-4 "Explain prompt caching"
llm -m zenmux/google/gemini-2.5-flash "Summarize this file" -f README.md

Options

OpenAI Chat Completions models inherit llm's built-in OpenAI options (temperature, top_p, max_tokens, json_object, etc.) and add:

Anthropic Messages models support:

Examples

JSON mode:

llm -m zenmux/openai/gpt-4o-mini -o json_object true "Return a JSON list of planets"

Structured output:

llm -m zenmux/openai/gpt-4o \
  --schema '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"}}}' \
  "Dr Who is 900 years old"

Extended thinking (Claude):

llm -m zenmux/anthropic/claude-sonnet-4 \
  -o thinking true -o thinking_budget 4000 -o max_tokens 8000 \
  "Prove there are infinitely many primes"

Reasoning effort (OpenAI o-series):

llm -m zenmux/openai/o3 -o reasoning_effort high "Design a B-tree index"

Streaming reasoning capture: When a model returns chain-of-thought, the plugin surfaces it as llm reasoning events — visible with llm -m zenmux/openai/o3 -t reasoning "…".

Image input:

llm -m zenmux/openai/gpt-4o -a photo.jpg "What's in this picture?"

Provider routing:

llm -m zenmux/openai/gpt-4o \
  -o provider '{"routing":{"type":"priority","providers":["openai","azure"]},"fallback":true}' \
  "Hello"

Features

• Unified access to all ZenMux-hosted models via OpenAI Chat Completions or Anthropic Messages APIs
• Automatic model discovery — live model list fetched and cached hourly; fallback list included if ZenMux is unreachable
• OpenAI surface extras: reasoning_effort, verbosity, logprobs/top_logprobs, stop, parallel_tool_calls, metadata, web_search_options, ZenMux provider routing, model_routing_config, reasoning
• Anthropic surface extras: extended thinking with configurable thinking_budget, top_k, stop_sequences, automatic prompt caching (5-minute ephemeral cache_control on system prompt and last user message)
• Extended thinking / chain-of-thought capture surfaced as llm reasoning stream events and persisted to response.response_json
• Attachment support derived from ZenMux input_modalities — vision models get image types, file-capable models get PDF, audio models get wav/mpeg; unknown modalities (e.g. video) are ignored
• Tool calling (function calling) support on both API surfaces
• Structured output — JSON mode and --schema-based structured output
• Async support — all model classes have AsyncChat / AsyncKeyModel counterparts
• llm zenmux models CLI command to list or refresh the cached model directory

License

Apache-2.0