orchestrator-mcp

Capability-routed MCP server: send each request to the model configured for that kind of work

Quick Start · How It Works · Tools · Consult · Configuration · Guardrails · Troubleshooting · Contributing

Research to one model, coding to another, cheap extraction to a third — and every answer comes back through the same validated envelope. Pointing a capability at your own deployment is a YAML edit. There is no code to change.

Since 0.2.0 there is a second path: consult routes a question to another vendor's coding agent — a Codex or Claude Code CLI on your machine, under its own account — so Claude Code can ask GPT and Codex can ask Claude, in either direction, with no second API key.

Published to PyPI as orchestrator-mcp-server — the shorter name is an empty registered project owned by someone else. The import package is orchestrator_mcp.

Quick Start

Write a config.yaml — start from config.example.yaml — and check that it loads:

ORCHESTRATOR_CONFIG=config.yaml uvx --from orchestrator-mcp-server python -c "from orchestrator_mcp.server import build_server; build_server(); print('config ok')"

A bad config fails here rather than at request time: every deployment must route to a declared capability, every capability must have a deployment behind it, and every fallback must name a real capability.

Claude Code

claude mcp add orchestrator --env ORCHESTRATOR_CONFIG=$PWD/config.yaml -- uvx orchestrator-mcp-server

Codex

In ~/.codex/config.toml:

[mcp_servers.orchestrator]
command = "uvx"
args = ["orchestrator-mcp-server"]
env = { ORCHESTRATOR_CONFIG = "/absolute/path/to/config.yaml" }

Both speak stdio, and every tool result is returned as structured content and as JSON text, so a client that reads only one of the two still gets the whole envelope.

Note Provider keys are read from the environment the server process gets, which is the client's environment — not your shell's. If a capability returns auth_failed while the same config works from a terminal, add the key to the client's env block.

Homebrew

brew tap crAK1644/tap
brew install orchestrator-mcp-server

That puts orchestrator-mcp-server on your PATH, so a client can call it directly instead of going through uvx:

claude mcp add orchestrator --env ORCHESTRATOR_CONFIG=$PWD/config.yaml -- orchestrator-mcp-server

Note Apple Silicon pours a prebuilt bottle. Everything else builds every Python dependency from source, including several Rust crates, which takes around fifteen minutes — uvx orchestrator-mcp-server is the same program from a prebuilt wheel in about a second.

From a checkout

uv sync && uv run pytest -q

claude mcp add orchestrator --env ORCHESTRATOR_CONFIG=$PWD/config.yaml -- uv run --directory $PWD orchestrator-mcp-server

Documentation

Everything lives in this file and in the annotated config. The links below are the fast path to a specific answer.

Getting started

• Quick Start — install into Claude Code or Codex
• Homebrew — brew tap crAK1644/tap
• Configuration — capabilities, deployments, limits
• config.example.yaml — a commented starting point
• System Requirements

Using it

• Tools — ask and list_capabilities
• The response envelope — what every call returns
• Error codes — the closed set callers branch on
• Consulting another agent — consult, and the CLI path
• The dashboard — read-only page over stored consultations

Understanding it

• How It Works — why capabilities and not a classifier
• Guardrails, and their limits — what is enforced, and what is not
• Not included — deliberate omissions

Development

• Tests — offline suite and the live smoke script
• Contributing · Releasing
• Troubleshooting · Bug Reports

How It Works

A capability is a LiteLLM model_name alias group. Several deployments can share one name, and litellm.Router already load-balances, retries, cools down, and falls back across them. So the routing engine is the config file:

model_list:
  - model_name: coding                  # capability, not a model
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY

  - model_name: coding                  # same capability, your own box
    litellm_params:
      model: openai/qwen-coder
      api_base: http://vllm.internal:8000/v1
      api_key: os.environ/LOCAL_VLLM_KEY

The pieces:

• The config file is the router. Capabilities, deployments, fallbacks, and retry policy are all LiteLLM's own schema, so litellm --config config.yaml runs on it unchanged.
• The caller states its capability. There is no intent classifier — the caller is already a language model and knows whether it is asking a coding question. Paying a second model to guess what the first one already knows buys a cost increase and a new failure mode.
• One envelope for every outcome. Success, refusal, truncation, and timeout all return the same shape, so callers branch on a field instead of parsing prose.

Tools

ask

list_capabilities

What each capability is for, the deployments behind it, and where it falls back.

The response envelope

{
  "ok": true,
  "content": "…",
  "data": null,
  "insufficient_context": false,
  "capability_requested": "coding",
  "model_used": "anthropic/claude-sonnet-4-5",
  "fallback_used": false,
  "finish_reason": "stop",
  "usage": { "prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30, "cost_usd": 0.0002 },
  "latency_ms": 412,
  "error": null
}

content holds the prose answer and is null in structured mode and on failure; data holds the validated object and is set only in structured mode; error is { code, message } whenever ok is false.

finish_reason rides along on failures too, when a provider replied at all — it is diagnosis rather than an answer, and length tells you to raise max_output_tokens instead of going looking for a bug. It is null when nothing was received.

Error codes

A closed set, so callers branch on a value instead of matching substrings.

error.message is bounded at 500 characters and never quotes the rejected output back at you.

Consulting another agent

ask talks to a model over HTTP. consult talks to an agent runtime — the codex or claude CLI installed on the same machine, launched as a subprocess, already logged into its own account. That difference is the whole point: it brings its own subscription, its own web search, and its own conversation state, so a multi-turn consultation with a different vendor's agent needs no key you do not already have.

Text goes out, structured text comes back. The consulted agent runs with its tools off, its sandbox read-only, and no MCP config, so it can answer and nothing else.

Enabling it

Two things: a consult: block (see config.example.yaml, which ships one commented out) and ORCHESTRATOR_HOST_RUNTIME, naming the runtime this installation is.

consult:
  database_path: ~/.orchestrator-mcp/consultations.sqlite3
  timeout_s: 180
  agents:
    codex-sol:
      runtime: codex
      command: codex
      model: gpt-5.6-sol
      priority: 10
      web_search: true
      scores: { coding: 95, research: 85, reasoning: 90 }

claude mcp add orchestrator --env ORCHESTRATOR_CONFIG=$PWD/config.yaml --env ORCHESTRATOR_HOST_RUNTIME=claude -- uvx orchestrator-mcp-server

The same server installed under Codex sets ORCHESTRATOR_HOST_RUNTIME=codex, and the routing table reverses itself. That variable is read from the environment and never from a tool argument: an agent that could name the host runtime could name someone else's and hand the work straight back to itself. Configure consult: without it and the server refuses to start.

No consult: block, no change: the server advertises exactly ask and list_capabilities, and a snapshot test asserts that schema byte for byte so the second path cannot reshape the first one under it. The snapshot is a regression guard generated from the current code, not a copy of 0.1.2's — the ask schema did move once between them, deliberately: response_schema's string arm now advertises maxLength: max_schema_chars, and that cap is enforced before the JSON is parsed rather than after. A 0.1.2 caller sending a schema inside the cap sees no difference.

Tools

consult arguments:

Routing is deterministic and operator-controlled: disabled agents are dropped, every agent whose runtime equals the host's is dropped, then score descending, priority ascending, id ascending. There is no fallback to a second candidate — if the chosen agent is not logged in, that is the answer you get, not a quiet substitution.

Keep the consultation_id

MCP gives a server no reliable identifier for the host's own conversation, so the returned consultation_id is the binding mechanism. Send it back on every later call about the same topic and the same native CLI session continues; omit it and each call starts a fresh one that remembers nothing. The tool description says so to the calling model, but a host that drops the field will silently get single-turn behaviour.

A consultation is pinned to the agent, runtime, and model it started on. Pointing a resumed one at a different target is session_target_mismatch, never a silent restart somewhere else.

The envelope

{
  "ok": true,
  "consultation_id": "6f1c…",
  "content": {
    "answer": "…",
    "assumptions": [],
    "uncertainties": ["whether the index is hot"],
    "follow_up_questions": ["how large is the table?"],
    "sources": [{ "title": "model", "locator": "internal", "source_type": "model" }]
  },
  "capability_requested": "coding",
  "source_mode_used": "model",
  "route": { "agent_id": "codex-sol", "runtime": "codex", "model": "gpt-5.6-sol",
             "capability_score": 95, "priority": 10, "explicitly_selected": false },
  "usage": { "prompt_tokens": 900, "completion_tokens": 120, "total_tokens": 1020 },
  "latency_ms": 8412,
  "error": null
}

Every content field is required, and arrays may be empty but never absent: "no assumptions" is a claim the consulted agent has to make, not one you have to infer from a missing key. When ok is false, content is null — a failed consultation never carries text that could read as an answer.

Consult error codes

What this path enforces

• It never consults itself. The host runtime is excluded at routing and refused on resume, so a stored consultation cannot become a loop back into the caller.
• No silent substitution. Not of the agent — there is no fallback candidate — and not of the model: both adapters check what the CLI reports it actually used against what you configured, and disagreement is configured_model_unavailable. Names are compared as token runs, so claude-sonnet-4 does not match claude-sonnet-4-5 and gpt-5.1 does not match gpt-5.10; a dated snapshot of the version you pinned does match, and a bare alias like opus has no version to disagree with.
• The consulted agent may not act. Codex runs --sandbox read-only --ignore-user-config --ignore-rules with approval_policy="never", shell, and subagents off — as -c config keys, because a 0.146 build rejects --ask-for-approval on exec outright; Claude Code runs --safe-mode --strict-mcp-config --tools "". Any action event in the stream fails the turn closed rather than being ignored.
• No credentials, ever. Preflight reads one bit: codex login status's exit code, or the loggedIn boolean out of claude auth status --json. Codex's login output is not even captured, and no part of either payload has a column to be stored in. Authentication is something you do in your own terminal — the server only ever tells you which command to run.
• Web mode is bounded by turns, not just by time. Claude Code has no --max-turns, so the event stream is counted here and the child is killed past web_turn_limit. The turn that spends the last of the budget is allowed to finish and answer; the one after it is not, and a consultation stopped there returns an error rather than an answer assembled from partial turns.
• No shell. Every invocation is create_subprocess_exec with an argument list, and the prompt travels over stdin, so there is neither an injection surface nor an argv ceiling. Children get their own process group and a timeout kills the group.
• Failures are envelopes. Including the ones that are not this server's fault.

The database at database_path holds every prompt and every answer verbatim — directory 0700, file 0600, set before anything is written. Set store_full_content: false to keep metadata and routing only. Nothing from a subprocess environment is stored at all.

The dashboard

Off by default. dashboard.enabled: true, then:

ORCHESTRATOR_CONFIG=config.yaml orchestrator-mcp-dashboard

A read-only page over the same SQLite file: agents and their last status check, the last 200 consultations, and per turn the compiled prompt, the answer, latency, usage, and any error. It opens the database mode=ro, serves GET and nothing else, refuses to bind anywhere but loopback, and rejects a request whose Host is not one — because what it serves is every prompt you have ever sent.

System Requirements

• Python 3.11, 3.12, or 3.13
• uv — or any installer, if you would rather use pip
• An MCP client that speaks stdio (Claude Code, Codex, or your own)
• At least one provider you hold credentials for, or a local endpoint
• Network access to whatever your config.yaml points at — nothing else phones home
• For consult only: the codex and/or claude CLI installed and logged in, and ORCHESTRATOR_HOST_RUNTIME set. Nothing else changes if you leave that block out.

A local Ollama endpoint works and costs nothing, which makes it a reasonable way to try the server before wiring up paid providers:

model_list:
  - model_name: fast
    litellm_params:
      model: ollama_chat/qwen2.5:7b
      api_base: http://localhost:11434

Configuration

ORCHESTRATOR_CONFIG points at the file; it defaults to config.yaml in the working directory. Keep it out of version control — it holds your endpoints.

capabilities:
  coding: "Writing, refactoring, reviewing, and debugging code."
  fast: "Cheap, low-latency answers. Classification, extraction, short replies."

model_list:
  - model_name: coding
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY

  - model_name: fast
    litellm_params:
      model: anthropic/claude-haiku-4-5-20251001
      api_key: os.environ/ANTHROPIC_API_KEY

router_settings:
  num_retries: 2
  cooldown_time: 60
  fallbacks:
    - coding: [fast]

Keys are referenced as os.environ/NAME and read at request time. Never inline the value.

Limits

Every one of these is a boundary the caller cannot cross, checked before a provider is called. A nonsensical value fails at startup rather than mid-request.

Guardrails, and their limits

This server sees a prompt and a completion. It has no ground truth, so it cannot verify factual claims, and nothing here should be read as a hallucination detector. What it does enforce:

• Shape is validated, not assumed. Structured replies are checked against your schema locally with jsonschema, regardless of whether the provider claims to enforce response_format. A violation is a failure, not a payload.
• Bounded repair. An invalid structured reply gets limits.schema_repair_attempts retries carrying the validator's complaint, then fails as schema_validation_failed. Never a best-effort half-parsed object.
• An unfinished answer is a failure, not a short answer. A completion cut off by the token limit comes back as output_truncated with content: null, and one the provider filtered as content_filtered. Neither is returned as prose, because a half answer reads exactly like a whole one. The same applies to a reply that stopped to call a tool (this server exposes none) and to the provider reasons LiteLLM normalizes to a plain stop — a malformed function call, an unspecified reason — which are read from the native reason it keeps alongside.
• The error tells you what broke, not what the model wrote. error.message gives the failing path and constraint (schema violation at answer/city: failed the 'maxLength' constraint) and is capped at 500 characters. The rejected value itself goes only back to the model that produced it, in the repair turn. Anything credential-shaped in that message — an API key, a Bearer header, a PEM block a provider quoted back — is replaced with [redacted] on both paths before the envelope leaves. It is a second line of defence and not a guarantee: a secret with no recognizable shape survives it.
• request_timeout_s bounds the call. Retries, cross-capability fallback, and repair turns all spend from one budget, so 120 cannot become 360. Each leg is given slightly less than what is left of that budget, so a hung deployment times out inside LiteLLM — which counts the failure and cools it down — instead of being cancelled from outside, which would leave the next request to pick it again. What it does not bound is CPU spent inside this process: validating a reply against a caller-supplied response_schema runs jsonschema, and a pathological pattern in that schema can backtrack for far longer than the deadline. re holds the GIL, so no timeout and no thread rescues it. max_schema_chars limits the size of what a caller can send, not what it can cost — treat response_schema as trusted input.
• Abstention is typed. With context set, the model is given an explicit way to say the material does not support an answer; it arrives as insufficient_context, not as prose you have to pattern-match.
• The server never ghostwrites. When ok is false, content and data are both null. It will not put a "Sorry, I couldn't…" string where a model's answer goes, because callers cannot tell those apart. Enforced by an assertion on every response and covered by tests.
• Degradation is visible. fallback_used and model_used always ride along, so an answer served by the backup after the primary died never passes as the intended one.
• The caller cannot smuggle a model. There is no free-form model parameter, only the capability enum. Routing stays operator-controlled.
• Boundaries reject early. Unknown capability, oversized prompt or system, empty prompt, and a malformed or oversized response_schema all fail before a provider is called.

Three known gaps. The MCP SDK drops unknown arguments before the handler sees them, so an unrecognized key is ignored at the protocol layer rather than rejected — direct calls into Orchestrator.ask do reject it. A response_schema containing a pathological pattern can burn CPU on the event loop during validation: the schema is size-capped but not analyzed, so treat schema authorship as a trusted operation. And when a provider call raises, error.message carries the first 500 characters of the provider's own exception text, which some providers fill with the API base, the offending request fragment, or a credential's environment variable name — and during a repair turn the request contains the model's previously rejected output. That text is diagnosis worth keeping for a caller that already reads your config; it is not safe to forward somewhere your config is not.

Tests

uv run pytest -q

315 tests, no network and no CLI installed. Deployments are stubbed with LiteLLM's mock_response, and the shapes it cannot express (no choices, null content, a truncated or filtered reply) are stubbed as raw ModelResponse objects. Includes the rate-limit-then-fallback path and the cooled-down-group path.

The consult path is stubbed the same way: codex and claude are replaced with small Python scripts on PATH that replay recorded event streams — success, resume, a substituted model, a tool-use event, a stream that never ends, one that ignores SIGTERM so the process-group kill has to be proven, and one that exits at once leaving a grandchild behind, because a reaped leader is exactly the case a kill routed by pid lookup misses. Cross-process behaviour (leases, migrations) is tested with real OS processes rather than tasks, because an in-process lock would pass either way.

Because all of that is stubbed, it proves this server's logic and nothing about your providers or your CLIs. For those:

uv run python smoke_live.py

Real calls against your config.yaml, roughly four short ones per capability, so it costs a little money — run it deliberately, not in CI. It checks the things only a live endpoint can answer: whether response_format survives the round trip, whether the model honours the abstention path instead of inventing, and what the provider actually sends as finish_reason when it runs out of room. Name capabilities as arguments to check only some (uv run python smoke_live.py fast).

ORCHESTRATOR_HOST_RUNTIME=claude uv run python smoke_consult_live.py

The same deal for the consult path, and the only thing that confirms the real CLI flags and event names: it needs both CLIs installed and logged in, and it spends whatever your subscriptions charge. Also not for CI.

Troubleshooting

Consult path:

Bug Reports

Open an issue with the envelope you got back — it carries error.code, model_used, fallback_used, finish_reason, and latency_ms, which is most of a diagnosis already. Add your config.yaml with the keys removed.

For anything that looks like a routing or retry problem, LiteLLM's own trace is the useful attachment:

LITELLM_LOG=DEBUG uv run python smoke_live.py 2>debug.log

It writes to stderr only, so stdout stays clean and the MCP stream is unaffected.

Contributing

Issues and pull requests are welcome. The bar for a change is a test that fails without it — the suite runs offline, so there is no key to obtain and no cost to pay. Keep config.yaml out of your commits.

1. Fork and branch.
2. Make the change.
3. Add the test that fails without it.
4. uv run pytest -q.
5. Open the pull request.

If you are adding a capability to your own setup, you do not need a pull request: it is a model_list entry.

Releasing

Bump version in pyproject.toml, then publish a GitHub Release tagged vX.Y.Z. release.yml runs the suite, checks the tag against the packaged version, and uploads to PyPI.

There is no API token in this repo. PyPI is configured as a Trusted Publisher for this workflow, so it mints a short-lived credential from the job's OIDC identity — nothing to store, nothing to rotate, nothing to leak.

Once the release is on PyPI, the tap needs the new version too. In a branch of that repo, point the formula's url and sha256 at the new sdist and delete the bottle do block — its root_url names the previous version's release, so leaving it sends installs to a tarball that is not there. Open a pull request: brew test-bot builds the bottle and uploads it as an artifact. Then run the brew pr-pull workflow with that pull request's number, which writes the new bottle do block, attaches the bottle to a release named orchestrator-mcp-server-X.Y.Z, and pushes to the tap's main.

Bottles are Apple Silicon only. x86 macOS reports "unbottled dependencies, so a bottle will not be built" — homebrew-core no longer bottles parts of this dependency tree for it — and both it and Linux build from source instead.

Not included

Semantic/embedding routing and RouteLLM-style predictive routing (the caller states its capability); Redis-backed distributed cooldown state (single process — LiteLLM enables it via config when you need a second node); streaming (MCP tool results return whole); sampling/createMessage loops; PII redaction and telemetry callbacks (available as LiteLLM callbacks when a requirement names one).

License

MIT — see LICENSE. Use it, fork it, ship it in something commercial. The only thing it asks is that the copyright notice travels with the code.

Support

• Issues — bugs and feature requests
• PyPI — releases
• LiteLLM docs — routing, fallbacks, and cooldowns
• Model Context Protocol — the protocol itself

Built on LiteLLM, Pydantic, and the Python MCP SDK.