Skip to main content

ModelMeld

Per-request capability routing for Claude Code, Cursor, Aider, Cline, Continue. Speaks OpenAI and Anthropic natively. Bring your own key — never stored.

License Python PyPI CI

Quickstart

Run the gateway yourself and route real OSS models in a couple of minutes. You bring a provider key (OpenRouter is the easiest — one key, many open-weight models, pay-as-you-go); ModelMeld picks the cheapest model that clears the quality bar per request.

pip install 'modelmeld[anthropic,openai]'
modelmeld setup --self-host

The wizard prompts for whichever provider keys you have (OpenRouter / Fireworks / Together for cloud-OSS routing, an optional Anthropic / OpenAI key for frontier escalation, or a local vLLM endpoint), enables capability routing, pre-configures Claude Code's /model picker, and — before declaring success — boots the gateway and proves a real OSS model served a request (it fails loudly rather than leaving you on a silent no-op).

Then, in two shells:

# shell 1 — run the gateway
source ~/.modelmeld/modelmeld-gateway.env
uvicorn modelmeld.api.server:app --host 0.0.0.0 --port 8080

# shell 2 — point Claude Code at it
source ~/.modelmeld/setup-claude-code.sh
claude

In the Claude Code TUI, type /model → pick ModelMeld — Auto (or Saver / Quality). No provider key yet? modelmeld setup --self-host --demo points you at the cheapest one-key on-ramp. Diagnose anytime with modelmeld doctor.

Hosted (managed) gateway — point your tool at our URL and skip running anything yourself. Currently invite-only beta: request access. Once you have a key, modelmeld setup --tool claude-code configures the hosted path.

Anthropic prompt caching, end-to-end

Anthropic prompt caching survives end-to-end through ModelMeld:

cache_control round-trip — first call writes 4933 tokens to cache; second call reads them back at 90% discount

Same payload sent twice, seconds apart. The second call hits Anthropic's prompt cache for 4933 tokens at 10% of normal rate — a 90% input-token discount, preserved through the gateway. Many other gateways strip cache_control markers; ModelMeld passes them through verbatim.

Why ModelMeld

You're paying frontier-model prices on every request — including the ones where a coding-tuned 7B model would produce identical output. Most gateways force a global choice ("use Anthropic" / "use OpenAI" / "use local"). ModelMeld picks per request, driven by a benchmark-weighted capability scout that knows which model class is sufficient for which task category.

How it compares

Capability ModelMeld Typical gateway
Anthropic cache_control preserved end-to-end ✅ ⚠️ Many strip it
Speaks /v1/chat/completions AND /v1/messages natively ✅ Usually OpenAI shape only
Audit headers expose the routing decision to the caller ✅ (x-modelmeld-routed-to, x-modelmeld-routed-model, etc.) Usually opaque
BYOK — keys never stored at rest ✅ Varies
Honest non-coverage list (what doesn't work yet) ✅ Rare

The differentiator isn't that we route. It's that we don't break the features upstream providers built into their APIs, and we tell you exactly what the gateway did with your request.

Three policies, three behaviors

Pick the policy that matches your work mode. They show up in any tool's /model picker as three options:

  • anthropic/modelmeld-saver — OSS-only. Never escalates to frontier. Predictable cost ceiling — you pay OSS-tier rates regardless of request complexity.
  • anthropic/modelmeld-auto — OSS by default; escalates to frontier (Sonnet/Opus) when the prompt contains 2+ reasoning markers ("think step by step", "explain your reasoning", etc.).
  • anthropic/modelmeld-quality — Frontier-first. Downgrades to OSS only on detected trivial work (autocomplete-shape, background calls).

Frontier-tier routing uses BYOK. Your Anthropic / OpenAI key is sent as a per-request header (x-modelmeld-byok-anthropic: sk-ant-…), used to make the upstream call, then forgotten. Never stored at rest, never logged.

Works with

Drop-in for any tool that speaks OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages.

Validated end-to-end: Claude Code · Codex CLI · opencode · Aider · AutoGen · CrewAI · LangGraph · OpenClaw · OpenAI SDK · anthropic-sdk-python · @anthropic-ai/sdk

Should work, not yet live-tested: Cursor · Cline · Continue

Frameworks can declare task category + agent role explicitly via x-modelmeld-task-category / x-modelmeld-agent-role headers — bypasses the classifier when your harness already knows what kind of work the request represents. See routing hints.

What doesn't work yet

Honest non-coverage list for the v1 OSS API surface:

  • Anthropic image content blocks (vision input) — deferred. Claude Code doesn't use vision; documented as a known gap rather than silently failing.

What's in the package

  • Three API surfaces, one routing pipeline. OpenAI-compatible at /v1/chat/completions (drop-in for any OpenAI-wire-format client), the OpenAI Responses API at /v1/responses (drop-in for Codex CLI), and Anthropic-compatible at /v1/messages (drop-in for Claude Code, anthropic-sdk-python, @anthropic-ai/sdk). All three stream via SSE, share the same router / memory / cache pipeline, and emit identical x-modelmeld-* audit headers.
  • Provider adapters — OpenAI, Anthropic (with full schema translation in both directions), vLLM, TensorRT-LLM. Each adapter retries transient errors (429 / 5xx / network blip) with exponential backoff.
  • Capability-based routing — CapabilityScout picks the cheapest model meeting a quality threshold for the prompt's task category, driven by the ModelRegistry.
  • Completion cache — exact-match (in-memory or Redis) + semantic (Qdrant); cache key pools across users routed to the same served model.
  • PII scrubbing — runs on every egress path before cloud upload.
  • Production-tuned defaults — full dev-tool detection catalog and a current default_registry.json snapshot ship as the defaults. All tunable via constructor args.

Self-host (manual config)

The Quickstart wizard is the recommended path — it writes the config below for you and verifies real routing. To wire it by hand:

The gateway reads MODELMELD_-prefixed env vars. Two settings are required for real routing — without them the gateway falls back to a no-op stub adapter that returns canned replies to every request:

pip install 'modelmeld[anthropic,openai]'

# 1. Capability routing (the scout picks a model per request).
export MODELMELD_ROUTING_POLICY=capability
# 2. At least one provider key (note the MODELMELD_ prefix):
export MODELMELD_OPENROUTER_API_KEY=sk-or-…   # cloud-OSS routing
# Optional frontier escalation for -auto / -quality:
export MODELMELD_ANTHROPIC_API_KEY=sk-ant-…
export MODELMELD_OPENAI_API_KEY=sk-…

uvicorn modelmeld.api.server:app --host 0.0.0.0 --port 8080

Then point your tool at http://localhost:8080. In self-host you supply upstream keys to the gateway via these env vars (not per-request BYOK headers — those are for the hosted multi-tenant path).

For routing across local vLLM + cloud providers, see docs/backends.md. For full self-host operational notes (TLS, scaling, observability), see docs/self-host.md.

Licensing — TL;DR

  • Code: AGPL-3.0-or-later. Use, modify, redistribute. Calling the gateway over HTTP from unmodified clients (Cursor, Aider, Claude Code, etc.) does NOT make those clients AGPL. Running a modified gateway as a service for third parties does require your modifications to also be AGPL.
  • Bundled snapshot data (scout/data/default_registry.json): CC-BY-4.0 with attribution. Use the snapshot scores anywhere.
  • Live curated registry feed (feed.modelmeld.ai): subscription product. Continuously updated, editorially weighted across multiple benchmark sources.

If you pip install modelmeld and never subscribe, everything works — you just route on a snapshot of benchmark data taken at OSS release date. Over ~6 months that snapshot stales relative to the current best-cost frontier; the feed is what keeps routing decisions sharp.

Full rationale, boundary contract, and commercial-licensing options: docs/license-rationale.md, docs/open-core-boundary.md, docs/registry-feed.md. Or email hello@modelmeld.ai.

Status

Production-ready for the routes documented here. Pre-1.0 on SemVer guarantees for the public Python API — see docs/api-stability.md for which symbols carry compatibility commitments and which are subject to change.

The HTTP surfaces (/v1/chat/completions, /v1/messages, the x-modelmeld-* audit headers) are stable in spirit; we won't break existing integrations without a major-version bump and a deprecation window.

Contributing

PRs welcome. See CONTRIBUTING.md for the dev workflow, code style (ruff format + ruff check + pyright), and DCO commit-signoff requirement. Issues labeled good first issue are intentionally scoped for first-time contributors.

We do not accept PRs that modify the bundled snapshot data files (scout/data/) — those are curated centrally for the live feed. File issues against bad routing decisions you observe and we'll evaluate adjustments for the next feed release.

Community

  • GitHub Issues — bugs + feature requests (after reading CONTRIBUTING.md)
  • GitHub Discussions — questions, ideas, integration help
  • Security — see SECURITY.md; report to security@modelmeld.ai (90-day disclosure window)

Enterprise tier

For production deployments needing API-key auth, RBAC, OIDC SSO, Postgres-backed SOC2-grade audit logs, encryption-at-rest, per-tenant rate limiting, FinOps dashboards, multi-tenant Qdrant cache, or the managed hosted tier — email hello@modelmeld.ai.

License

Metadata

Release files for modelmeld 0.18.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for modelmeld 0.18.0
File Size Uploaded Provenance Actions
modelmeld-0.18.0.tar.gz 219.7 kB With provenance Details for modelmeld-0.18.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for modelmeld 0.18.0
File Interpreter ABI Platform Actions
modelmeld-0.18.0-py3-none-any.whl Python 3 none any Details for modelmeld-0.18.0-py3-none-any.whl

Total release size: 492.2 kB

Release files / modelmeld-0.18.0.tar.gz

Download URL modelmeld-0.18.0.tar.gz
Size 219.7 kB
Tags Source
SHA-256 checksum
How to use checksums
1ed66a6f97eadbb81878d38d08fe54d573614b059ea8f4d5d8025dd3d15b3338
BLAKE2b-256 checksum
How to use checksums
9df1f683dcde5b146a9744660d1601617ef4d2595b4bddbf86030aa01c930c87
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 23, 2026.

Transparency log

Release files / modelmeld-0.18.0-py3-none-any.whl

Download URL modelmeld-0.18.0-py3-none-any.whl
Size 272.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ab1336e770240cc8ce829480a2bdc59dd29e8ee0d8b932e9da459a490d615d93
BLAKE2b-256 checksum
How to use checksums
d0cb1fcdec1e75c9a87cdc361cd96903c6ee9af7dbb763288862b2532f34a15f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 23, 2026.

Transparency log

Security for modelmeld 0.18.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for modelmeld-0.18.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 23, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.18.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page