LLM Market

LLM Market is an open, sequential router for LLM work. In Codex, a bundled state/action JEPA chooses the next bounded compute action at each meaningful event: accept, answer quickly, execute normally, plan, review or repair, rescue, or run a small council.

The point is simple: stop sending every prompt to the most expensive model when a cheaper route can match the answer.

Current Behavior

Codex routing is automatic and JEPA-first after a one-time install. Users ask a normal question; lifecycle hooks send local, transient input to a warm controller, and Codex privately applies the selected next action. There is no activation phrase, model chooser, execution contract, or per-turn setup.

The controller carries a six-state trajectory across prompts, tools, validation, workers, follow-ups, and compaction. A conclusive validator pass forces acceptance, a failure forbids acceptance, and hard limits prevent concurrent mutators or runaway escalation. Missing infrastructure fails open to ordinary Codex.

Standalone plan and run retain the general model-market surface for provider experiments. The retired whole-route Codex allocator, sticky epochs, and Gate 2 lifecycle are no longer shipped or callable.

Training runs, raw prompts, provider output, and local evidence stay outside the package. The wheel contains only sanitized inference assets: the checkpoint, local embedder, consequence heads, scalers, schema, thresholds, and provenance.

Quickstart

For automatic Codex routing, hand Codex docs/LLM_MARKET_INSTALL.md. It installs the exact current PyPI release into an isolated environment, installs the bundled skill and agents, checks the integration, and runs the packaged hook-to-controller smoke. No source checkout, personal code history, personal JEPA, API key, model selection, or activation phrase is required.

For the standalone model-market CLI:

python -m pip install llm-market
llm-market plan "Should this prompt use one model, bids, or a council?"

For local development:

python -m pip install -e ".[dev]"
python -m pytest

Inspect a route without calling a model:

llm-market plan "Should this prompt use one model, bids, or a council?"

Run locally with deterministic mock models:

llm-market run --mock --trace "Compare carbon taxes and cap-and-trade."

Mock answers are synthetic. Use them to inspect route mechanics, traces, and cost accounting before spending live provider calls.

The public package and CLI are llm-market. The Python import/module is llm_market because Python module names cannot contain hyphens, so python -m llm_market works too.

Run With Real Models

Codex CLI (Optional Standalone Provider)

This section is only for running llm-market run --provider codex from a terminal. The in-app Codex skill and model-pinned agents described below do not require a separately installed Codex CLI.

Codex model ids use codex:<model-slug>:<reasoning-effort>, for example codex:gpt-5.6-luna:medium, codex:gpt-5.6-terra:high, or codex:gpt-5.6-sol:max. Supported effort values are low, medium, high, xhigh, max, and ultra; the selected Codex model determines which values are available. The adapter shells out to an authenticated local codex exec.

Plan with the estimated-cost Codex cards:

llm-market plan \
  --config examples/codex_estimated_cost_models.json \
  "Compute 19 * 23 and give only the answer."

Run a live Codex call:

llm-market run \
  --provider codex \
  --config examples/codex_estimated_cost_models.json \
  --max-tokens 140 \
  "Compute 19 * 23 and give only the answer."

By default, Codex provider runs ignore user-level Codex config so evals stay reproducible. Pass --codex-use-user-config when you want the user's local Codex settings involved. Pass --codex-oss when you intentionally want Codex OSS or local-provider mode, such as Ollama or LM Studio.

Automatic Codex Next Actions

For an external-user installation, use the self-contained Codex install handoff. It installs LLM Market as an isolated application under ~/.local/share/llm-market, exposes the normal llm-market command through ~/.local/bin, and does not modify a general Python environment. If that launcher directory is not already on PATH, the handoff completes the installation and reports exact current-shell and persistent shell-profile activation commands without editing shell startup files itself.

Once installed:

llm-market codex-install
llm-market codex-install --check

In Codex, open /hooks, trust all eight LLM Market entries, and restart. The collapsed rows are app-owned Hook 1 labels; expand them to see the distinct LLM Market: ... descriptions. Submit one ordinary prompt, then run llm-market codex-doctor --smoke; it reports passed_with_warnings until that real automatic route has been observed. From then on, a normal user request is enough. The hook starts a warm local controller in the background, routes each meaningful lifecycle event, and adds private delegation context only when another model action is useful.

The default roles are:

• Luna Medium for simple answers and bounded follow-ups;
• Terra High for normal coding, GitHub/PR/CI work, repair, and tests;
• Sol High for important read-only planning, architecture, research, and review;
• Sol Max for repeated-failure rescue and council synthesis.

A council is a read-only Terra plus Sol High panel with Sol Max synthesis. One mutator remains responsible for workspace changes. Objective validation overrides learned estimates, automatic work is capped at six direct worker actions plus one council, and any controller failure leaves native Codex unchanged.

The bundled v3 checkpoint is a real context-6, horizon-2 StructuredStateJEPA. Its low-weight synthetic bootstrap contains 13,090 transitions across 240 grouped semantic bases; it is a useful starting policy, not evidence about live model quality. Packaging is blocked unless the latent predictor and all seven action-conditioned consequence targets beat their held-out baselines. Run python tools/evaluate_next_action_jepa.py for the fixed role/lifecycle probes.

No prompt or answer text is persisted. The local controller stores only allowlisted embeddings, hashes, counters, decisions, and outcome signals. Use llm-market codex-install --remove-hooks to remove only LLM Market's hooks, or llm-market codex-uninstall to remove the complete managed Codex integration while preserving unrelated skills, agents, and hooks. The doctor verifies the package, installed assets, controller route, and prompt-persistence boundary; Codex hook trust and live model entitlement remain manual checks. See Automatic Codex Next-Action Routing.

Cursor Agent CLI

Cursor model ids use cursor:<cursor-model-slug>, for example cursor:gemini-3.1-pro or cursor:claude-4.6-opus-high. The adapter shells out to an authenticated local cursor agent --print run.

Check Cursor readiness:

cursor agent status
cursor agent models

Run live through Cursor's own model surface:

llm-market run \
  --provider cursor \
  --config examples/cursor_models.json \
  --max-tokens 400 \
  --trace \
  "Review this patch plan and say what is missing."

If you omit --config, --provider cursor uses a built-in starter Cursor model universe with Composer, Gemini, and Claude/Opus cards, not Cursor GPT/OpenAI cards. The starter costs are relative routing weights because Cursor does not expose stable public per-token prices through this CLI path. Replace them with your own observed costs, latency, and quality data when you have it. The starter Cursor cards include request_overhead_tokens: 60000, because Cursor Agent runs can report a large hidden prompt/workspace overhead even for short user prompts. Replace this reserve with your own observed overhead for your repo and Cursor workspace.

Cursor runs default to --cursor-mode ask and pass --trust so unattended skill runs do not stop at workspace-trust prompts. Use --cursor-mode plan for Cursor's planning mode, --cursor-no-trust when you want Cursor to enforce its trust prompt, and --cursor-sandbox enabled|disabled when you want to pass Cursor Agent's sandbox flag explicitly.

Mixed Local Agents

Use --provider multi when one run should mix local providers. The multiplexer dispatches by model id:

• codex:* goes to Codex CLI
• cursor:* goes to Cursor Agent CLI
• other OpenRouter-style ids go to OpenRouter

With no config, --provider multi uses the built-in Codex + Cursor starter cards:

llm-market run \
  --provider multi \
  --force-council \
  --max-tokens 500 \
  --trace \
  "Have a small council review this architecture choice."

Pass --config when you want a custom mixed universe or want to add OpenRouter models. OpenRouter calls still need OPENROUTER_API_KEY, but the multiplexer does not require that key unless an OpenRouter-style model is actually called.

OpenRouter

Set OPENROUTER_API_KEY, then use an OpenRouter model-card config:

export OPENROUTER_API_KEY=...

llm-market run \
  --provider openrouter \
  --config examples/openrouter_models.json \
  --trace \
  "Find the best route for this code-review prompt."

Codex Skill

This repo includes a Codex skill in skills/llm-market. Install that folder as ~/.codex/skills/llm-market to call LLM Market from any Codex thread:

python tools/install_integrations.py codex
python tools/install_integrations.py codex --check

The in-app skill uses Codex custom agents directly. Do not install, update, log in to, or invoke a separate codex executable just to use this path.

The installer honors --codex-home first, then CODEX_HOME, then defaults to ~/.codex. It deletes stale installed files that no longer exist in the repo, preserves local INSTALL_RECEIPT.txt, installs eight model-pinned profiles for four models and automatic lifecycle hooks, verifies the installed copy, and runs a no-spend wrapper smoke. Restart Codex after a sync so changed skill and hook metadata is picked up.

The skill wrapper resolves LLM Market in this order:

1. an explicit --repo path
2. LLM_MARKET_REPO
3. the current repo or a parent directory
4. the skill directory or a parent directory
5. an installed Python package for plain CLI calls
6. a cached GitHub checkout when repo assets are requested or --update-github is used

If no checkout is found, the wrapper can clone the GitHub repo into $CODEX_HOME/llm-market/repo and run from that cached copy:

python ~/.codex/skills/llm-market/scripts/llm_market_cli.py \
  --update-github \
  plan \
  "Should this prompt use one model, bids, or a council?"

Inside Codex, plain skill calls default to a skill-local Codex-only model-card file, so they do not select OpenRouter or other non-Codex model ids unless you explicitly pass --config or an explicit non-Codex provider. Plain skill live-eval calls also default to --provider codex. Use --config when you intentionally want a custom, OpenRouter, or server-side provider universe.

Run the read-only provider doctor before Codex local-provider work:

python ~/.codex/skills/llm-market/scripts/codex_provider_doctor.py

The skill can distribute the package, examples, eval packs, and wrapper from GitHub. Live calls still need local provider state: Codex installed and authenticated, Cursor CLI installed and authenticated, or OpenRouter/OpenAI- compatible API keys. The skill reports local provider readiness; it does not edit ~/.codex/config.toml.

Check all local integration surfaces from the repo:

python tools/install_integrations.py all --check

Today this means a Codex skill sync/check, a no-spend Cursor provider plan smoke using the built-in Cursor starter cards, and an explicit Claude Code pending notice. Claude models are available through OpenRouter-style cards and Cursor cards; a Claude Code local-provider adapter is not checked in yet.

The public, self-contained installation path is the PyPI wheel plus docs/LLM_MARKET_INSTALL.md. Source installs and the wrapper's GitHub-checkout fallback are maintainer/development paths; they are not prerequisites for automatic Codex routing.

How Routing Works

For automatic Codex sessions, the default path is sequential:

1. The local MiniLM encoder turns the transient prompt or answer plus recent sanitized trajectory into a state embedding.
2. StructuredStateJEPA consumes up to six states and one candidate action, predicts the next latent state, and applies learned consequence heads for quality, success, progress, rescue, unnecessary continuation, latency, and cost.
3. The deterministic controller removes actions that violate authority, validation, concurrency, model availability, or compute limits.
4. It chooses the cheaper action among predictions within 0.03 expected quality of the best, with stricter lift gates for Sol Max and councils.
5. Hooks privately ask Codex to accept, delegate, run a bounded council, or continue natively. The next lifecycle event closes the loop.

The controller chooses one bounded next action, not one model for the entire task. It makes a fresh choice when the user submits a message and when Codex reaches a Stop boundary. Tool, worker, validation, and compaction events update the trajectory in between; compaction does not delay or trigger routing.

flowchart TD
    A["User submits a message"] --> B["Encode the message and restore recent task state"]
    B --> C["JEPA predicts consequences for every legal next action"]
    C --> D["Apply authority, validation, concurrency, and compute gates"]
    D --> E{"Choose the next bounded action"}

    E -->|answer_fast| F["Luna Medium answers"]
    E -->|execute_standard| G["Terra High inspects or implements"]
    E -->|plan_important| H["Sol High plans"]
    E -->|review_or_repair| I["Sol High reviews a failure"]
    E -->|rescue_max| J["Sol Max handles repeated failure"]
    E -->|council| K["Read-only council with Sol Max synthesis"]
    E -->|continue_native| L["Parent Codex continues"]
    E -->|accept_current| M["Finish the task"]

    F --> N["Tool, worker, and validation events update state"]
    G --> N
    H --> N
    I --> N
    J --> N
    K --> N
    L --> N

    N --> O["Codex reaches a Stop boundary"]
    O --> C
    P["PreCompact / PostCompact"] -. "Record state only; no route choice" .-> N
    Q["User follow-up"] --> B

A conclusive validation pass ends the loop. A first conclusive failure makes a Sol High review the legal repair action; repeated failure can unlock Sol Max. When multiple actions have nearly equal predicted quality, the cheaper one wins.

The JEPA receives state and action descriptors; it does not receive a hand- written “easy coding” or “hard research” route rule. Deterministic parsing is used only to preserve the user's answer/read/write authority. Standalone market plan and run remain separate provider-experiment tools; they do not control the automatic Codex lifecycle.

Configuration

Model cards are editable beliefs about cost, latency, reliability, context, and capability:

{
  "models": [
    {
      "id": "openai/gpt-latest",
      "provider": "openai",
      "capabilities": {
        "coding": 0.94,
        "reasoning": 0.92,
        "writing": 0.82
      },
      "input_cost_per_mtok": 5.0,
      "output_cost_per_mtok": 15.0,
      "context_tokens": 256000,
      "latency_ms": 2500,
      "reliability": 0.94,
      "tags": ["structured", "tool-use", "generalist"]
    }
  ]
}

Evidence rows record what worked on prior prompts:

{"model":"openai/gpt-4.1-mini","dimensions":{"coding":1.0},"score":1.0,"correct":true,"complexity":0.32,"risk":0.12}

Exact or strongly similar prompt history counts more than broad task-family averages. A single model needs enough margin before it can suppress council routing.

Python API

import asyncio

from llm_market import HeuristicMarketJudge, MarketEngine, MockModelClient
from llm_market.presets import default_model_cards

client = MockModelClient.from_static(
    {
        "openai/gpt-latest": "Strong coding and structured reasoning answer.",
        "anthropic/claude-opus-latest": "Careful critique and risk analysis.",
        "google/gemini-pro-latest": "Broad research and long-context synthesis.",
    }
)

engine = MarketEngine(
    model_cards=default_model_cards(),
    client=client,
    judge=HeuristicMarketJudge(),
)

result = asyncio.run(engine.run("Compare two approaches to a risky migration."))
print(result.answer)
print(result.trace.route.to_dict())

CLI Reference

next-action inspects the automatic Codex controller without exposing raw conversation text:

llm-market next-action --input route_state.json --format json

next-action-serve runs the warm local Unix-socket process used by hooks. The installer starts it automatically when needed.

plan explains the route without model calls:

llm-market plan --format json \
  --expected-output-tokens 800 \
  --max-expected-cost-usd 0.02 \
  "Write a SQL query for weekly retention."

llm-market plan --provider cursor \
  "Should Cursor handle this prompt with one model or a council?"

llm-market plan --provider multi \
  "Should local Codex and Cursor agents review this architecture together?"

Installed-package fallback is intentionally narrower: it can run the CLI and default in-package model cards, but repo assets such as examples/, outputs/, and skills/ require a source checkout or the cached GitHub checkout. The real-model commands below assume you are in a source checkout or the skill has resolved a cached GitHub checkout.

run executes a route:

llm-market run --mock --trace "Should this question use a council?"
llm-market run --mock --market auto --trace "Compare these migration options."
llm-market run --mock --council off --trace "Force single-model routing."

Presets set the operating posture without changing model cards:

• --preset fast: single-model, no bids, no council; uses evidence and cost pressure to prefer cheap earned routes.
• --preset balanced: default evidence/cost-aware routing.
• --preset max: bid-first and more quality-seeking; allows larger councils, one fallback, and lower cost/latency penalties without blindly forcing a council.

llm-market run \
  --provider codex \
  --config examples/codex_estimated_cost_models.json \
  --evidence ~/.llm-market/evidence.jsonl \
  --preset fast \
  "Summarize this diff and flag likely bugs."

live-eval calls real providers and compares the routed answer with baselines:

llm-market live-eval \
  --provider codex \
  --config examples/codex_estimated_cost_models.json \
  --evidence ~/.llm-market/evidence.jsonl \
  --tasks examples/codex_complex_multistep_tasks.jsonl \
  --baseline-mode external \
  --baseline-config examples/codex_complex_baselines.json \
  --provider-timeout-seconds 240 \
  --max-tokens 700 \
  --progress \
  --output ~/.llm-market/runs/new_codex_run.json

Use --baseline-mode external --baseline-config ... when you want a clean comparison against direct baselines such as xhigh-for-everything and cheap fixed-single arms without direct-running every selectable candidate. Omit --baseline-mode to keep the default full-baseline behavior.

For objective eval tasks with graders, live-eval also has an experimental validation ladder. It is off by default. When enabled, the routed answer is graded first; only a failed grade can trigger same-route retry or forced-council escalation:

llm-market live-eval \
  --provider codex \
  --config examples/codex_estimated_cost_models.json \
  --tasks examples/codex_complex_multistep_tasks.jsonl \
  --baseline-mode none \
  --validation-ladder same-route-then-council

Use --best-single auto for product routing. When similar prior outcomes show that a selectable model repeatedly dominates, the router can use that model directly.

evidence-ingest turns live-eval artifacts into future routing evidence:

llm-market evidence-ingest \
  --results ~/.llm-market/runs/<new-run>.json \
  --output ~/.llm-market/evidence.jsonl

Evidence can include prompt text, run identity, latency, route metadata, and model outcomes. Redact or keep private any eval artifact that contains customer, repo, or personal task text before publishing an evidence store.

route-episodes export preserves richer scored routing decisions for local inspection, shadow evaluation, and future calibration or training:

llm-market live-eval \
  --provider codex \
  --config examples/codex_estimated_cost_models.json \
  --tasks examples/codex_complex_multistep_tasks.jsonl \
  --baseline-mode selectable \
  --output ~/.llm-market/runs/<run-id>.json

llm-market route-episodes export \
  --results ~/.llm-market/runs/<run-id>.json \
  --output ~/.llm-market/route_episodes.jsonl

Compact evidence JSONL is the runtime prior for today's router. Route episodes JSONL is a richer private artifact with prompt profile, candidates, policy, route action, scored outcome, baselines, and counterfactuals. Route episodes are local/private by default: LLM Market does not upload, sync, phone home, collect telemetry, or share them automatically. Use --redact-prompt when exporting a dedupable prompt-hash-only copy. Future global route models should be trained only from maintainer-controlled, public/shareable, or explicit opt-in artifacts.

The only bundled Codex routing model is builtin:codex-next-action-jepa-v3. Older whole-route, sticky-epoch, route-world, sufficiency, and Gate 2 artifacts are not included in the wheel. The generic market CLI still accepts explicit local research advisors by path, but none participate in automatic Codex routing.

Offline ability-space research

The standalone research CLI can place a model into an explicit local ability-space artifact from graded episodes:

llm-market route-world onboard-model \
  --ability-space runs/my_ability_space.json \
  --model-id my-new-model \
  --episodes runs/my_new_model_episodes.jsonl \
  --output runs/my_new_model_placement.json

No ability-space model is bundled or used by automatic Codex routing. See docs/model_onboarding.md for the local-artifact workflow.

Repo Map

• src/llm_market/analysis.py: transparent profiling for standalone market experiments.
• src/llm_market/routing.py: standalone provider-market route selection.
• src/llm_market/bidding.py: model self-bids for market routing.
• src/llm_market/evidence.py: prior-result observations and evidence deltas.
• src/llm_market/route_episodes.py: richer local route episode export schema.
• src/llm_market/next_action.py: automatic Codex state/action contracts and policy.
• src/llm_market/next_action_server.py: local lifecycle controller.
• src/llm_market/route_world.py: explicit local research-advisor runtime.
• src/llm_market/sufficiency.py: explicit local sufficiency experiments.
• src/llm_market/engine.py: route execution and fallback behavior.
• src/llm_market/judge.py: heuristic judge and LLM judge.
• src/llm_market/clients.py: OpenRouter/OpenAI-compatible client and Codex CLI client.
• examples/codex_estimated_cost_models.json: cost-aware Codex model cards.
• examples/codex_complex_multistep_tasks.jsonl: harder rubric-graded Codex eval tasks.
• examples/codex_complex_baselines.json: direct xhigh and cheap-single baselines for complex-task evals.
• skills/llm-market: Codex skill wrapper and integration notes.
• tools/install_integrations.py: sync/check local integration surfaces.

OpenRouter Fusion Relationship

OpenRouter Fusion made the council primitive legible: ask multiple models, have a judge compare answers, then produce one response. LLM Market keeps that primitive open and adds the route policy around it: when to pay for a council, when to ask for bids, when a cheap model is enough, when prior evidence should promote one model, and what trace data should be saved for the next prompt.

Scope

• Codex cost comparisons use estimated relative costs because Codex CLI does not expose per-call billing.
• Provider-specific cache pricing, KV-cache accounting, and cache-hit discounts are future cost-model work.
• The included live result is a small complex-task eval. Run your own task families before trusting any routing policy in production.
• Model cards need current prices, reliability estimates, and observed evidence to stay useful.

Third-Party Notices

See THIRD_PARTY_NOTICES.md.