modelmeld 0.18.0

AI gateway with capability-based routing across providers + bring-your-own-key passthrough. OpenAI- and Anthropic-compatible. Drop-in for Claude Code, Cursor, Aider, Cline, Continue.

ModelMeld

Per-request capability routing for Claude Code, Cursor, Aider, Cline, Continue. Speaks OpenAI and Anthropic natively. Bring your own key — never stored.

Quickstart

Run the gateway yourself and route real OSS models in a couple of minutes. You bring a provider key (OpenRouter is the easiest — one key, many open-weight models, pay-as-you-go); ModelMeld picks the cheapest model that clears the quality bar per request.

pip install 'modelmeld[anthropic,openai]'
modelmeld setup --self-host

The wizard prompts for whichever provider keys you have (OpenRouter / Fireworks / Together for cloud-OSS routing, an optional Anthropic / OpenAI key for frontier escalation, or a local vLLM endpoint), enables capability routing, pre-configures Claude Code's /model picker, and — before declaring success — boots the gateway and proves a real OSS model served a request (it fails loudly rather than leaving you on a silent no-op).

Then, in two shells:

# shell 1 — run the gateway
source ~/.modelmeld/modelmeld-gateway.env
uvicorn modelmeld.api.server:app --host 0.0.0.0 --port 8080

# shell 2 — point Claude Code at it
source ~/.modelmeld/setup-claude-code.sh
claude

In the Claude Code TUI, type /model → pick ModelMeld — Auto (or Saver / Quality). No provider key yet? modelmeld setup --self-host --demo points you at the cheapest one-key on-ramp. Diagnose anytime with modelmeld doctor.

Hosted (managed) gateway — point your tool at our URL and skip running anything yourself. Currently invite-only beta: request access. Once you have a key, modelmeld setup --tool claude-code configures the hosted path.

Anthropic prompt caching, end-to-end

Anthropic prompt caching survives end-to-end through ModelMeld:

Same payload sent twice, seconds apart. The second call hits Anthropic's prompt cache for 4933 tokens at 10% of normal rate — a 90% input-token discount, preserved through the gateway. Many other gateways strip cache_control markers; ModelMeld passes them through verbatim.

Why ModelMeld

You're paying frontier-model prices on every request — including the ones where a coding-tuned 7B model would produce identical output. Most gateways force a global choice ("use Anthropic" / "use OpenAI" / "use local"). ModelMeld picks per request, driven by a benchmark-weighted capability scout that knows which model class is sufficient for which task category.

How it compares

Capability	ModelMeld	Typical gateway
Anthropic cache_control preserved end-to-end	✅	⚠️ Many strip it
Speaks /v1/chat/completions AND /v1/messages natively	✅	Usually OpenAI shape only
Audit headers expose the routing decision to the caller	✅ (x-modelmeld-routed-to, x-modelmeld-routed-model, etc.)	Usually opaque
BYOK — keys never stored at rest	✅	Varies
Honest non-coverage list (what doesn't work yet)	✅	Rare

The differentiator isn't that we route. It's that we don't break the features upstream providers built into their APIs, and we tell you exactly what the gateway did with your request.

Three policies, three behaviors

Pick the policy that matches your work mode. They show up in any tool's /model picker as three options:

• anthropic/modelmeld-saver — OSS-only. Never escalates to frontier. Predictable cost ceiling — you pay OSS-tier rates regardless of request complexity.
• anthropic/modelmeld-auto — OSS by default; escalates to frontier (Sonnet/Opus) when the prompt contains 2+ reasoning markers ("think step by step", "explain your reasoning", etc.).
• anthropic/modelmeld-quality — Frontier-first. Downgrades to OSS only on detected trivial work (autocomplete-shape, background calls).

Frontier-tier routing uses BYOK. Your Anthropic / OpenAI key is sent as a per-request header (x-modelmeld-byok-anthropic: sk-ant-…), used to make the upstream call, then forgotten. Never stored at rest, never logged.

Works with

Drop-in for any tool that speaks OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages.

Validated end-to-end: Claude Code · Codex CLI · opencode · Aider · AutoGen · CrewAI · LangGraph · OpenClaw · OpenAI SDK · anthropic-sdk-python · @anthropic-ai/sdk

Should work, not yet live-tested: Cursor · Cline · Continue

Frameworks can declare task category + agent role explicitly via x-modelmeld-task-category / x-modelmeld-agent-role headers — bypasses the classifier when your harness already knows what kind of work the request represents. See routing hints.

What doesn't work yet

Honest non-coverage list for the v1 OSS API surface:

• Anthropic image content blocks (vision input) — deferred. Claude Code doesn't use vision; documented as a known gap rather than silently failing.

What's in the package

• Three API surfaces, one routing pipeline. OpenAI-compatible at /v1/chat/completions (drop-in for any OpenAI-wire-format client), the OpenAI Responses API at /v1/responses (drop-in for Codex CLI), and Anthropic-compatible at /v1/messages (drop-in for Claude Code, anthropic-sdk-python, @anthropic-ai/sdk). All three stream via SSE, share the same router / memory / cache pipeline, and emit identical x-modelmeld-* audit headers.
• Provider adapters — OpenAI, Anthropic (with full schema translation in both directions), vLLM, TensorRT-LLM. Each adapter retries transient errors (429 / 5xx / network blip) with exponential backoff.
• Capability-based routing — CapabilityScout picks the cheapest model meeting a quality threshold for the prompt's task category, driven by the ModelRegistry.
• Completion cache — exact-match (in-memory or Redis) + semantic (Qdrant); cache key pools across users routed to the same served model.
• PII scrubbing — runs on every egress path before cloud upload.
• Production-tuned defaults — full dev-tool detection catalog and a current default_registry.json snapshot ship as the defaults. All tunable via constructor args.

Self-host (manual config)

The Quickstart wizard is the recommended path — it writes the config below for you and verifies real routing. To wire it by hand:

The gateway reads MODELMELD_-prefixed env vars. Two settings are required for real routing — without them the gateway falls back to a no-op stub adapter that returns canned replies to every request:

pip install 'modelmeld[anthropic,openai]'

# 1. Capability routing (the scout picks a model per request).
export MODELMELD_ROUTING_POLICY=capability
# 2. At least one provider key (note the MODELMELD_ prefix):
export MODELMELD_OPENROUTER_API_KEY=sk-or-…   # cloud-OSS routing
# Optional frontier escalation for -auto / -quality:
export MODELMELD_ANTHROPIC_API_KEY=sk-ant-…
export MODELMELD_OPENAI_API_KEY=sk-…

uvicorn modelmeld.api.server:app --host 0.0.0.0 --port 8080

Then point your tool at http://localhost:8080. In self-host you supply upstream keys to the gateway via these env vars (not per-request BYOK headers — those are for the hosted multi-tenant path).

For routing across local vLLM + cloud providers, see docs/backends.md. For full self-host operational notes (TLS, scaling, observability), see docs/self-host.md.

Licensing — TL;DR

• Code: AGPL-3.0-or-later. Use, modify, redistribute. Calling the gateway over HTTP from unmodified clients (Cursor, Aider, Claude Code, etc.) does NOT make those clients AGPL. Running a modified gateway as a service for third parties does require your modifications to also be AGPL.
• Bundled snapshot data (scout/data/default_registry.json): CC-BY-4.0 with attribution. Use the snapshot scores anywhere.
• Live curated registry feed (feed.modelmeld.ai): subscription product. Continuously updated, editorially weighted across multiple benchmark sources.

If you pip install modelmeld and never subscribe, everything works — you just route on a snapshot of benchmark data taken at OSS release date. Over ~6 months that snapshot stales relative to the current best-cost frontier; the feed is what keeps routing decisions sharp.

Full rationale, boundary contract, and commercial-licensing options: docs/license-rationale.md, docs/open-core-boundary.md, docs/registry-feed.md. Or email [e-mail].

Status

Production-ready for the routes documented here. Pre-1.0 on SemVer guarantees for the public Python API — see docs/api-stability.md for which symbols carry compatibility commitments and which are subject to change.

The HTTP surfaces (/v1/chat/completions, /v1/messages, the x-modelmeld-* audit headers) are stable in spirit; we won't break existing integrations without a major-version bump and a deprecation window.

Contributing

PRs welcome. See CONTRIBUTING.md for the dev workflow, code style (ruff format + ruff check + pyright), and DCO commit-signoff requirement. Issues labeled good first issue are intentionally scoped for first-time contributors.

We do not accept PRs that modify the bundled snapshot data files (scout/data/) — those are curated centrally for the live feed. File issues against bad routing decisions you observe and we'll evaluate adjustments for the next feed release.

Community

• GitHub Issues — bugs + feature requests (after reading CONTRIBUTING.md)
• GitHub Discussions — questions, ideas, integration help
• Security — see SECURITY.md; report to [e-mail] (90-day disclosure window)

Enterprise tier

For production deployments needing API-key auth, RBAC, OIDC SSO, Postgres-backed SOC2-grade audit logs, encryption-at-rest, per-tenant rate limiting, FinOps dashboards, multi-tenant Qdrant cache, or the managed hosted tier — email [e-mail].

License

• Code: AGPL-3.0-or-later (see LICENSE, NOTICE, docs/license-rationale.md)
• Data files: CC-BY-4.0 (see scout/data/LICENSE.md)
• Live feed: subscription terms (see NOTICE and docs/registry-feed.md)