Skip to main content

The LLM Council

$ council run drafter --mode arch "Design a mass hallucination prevention system"

                    ╔══════════════════════════════════════════════════════════╗
                    ║             ⚖️  THE LLM COUNCIL CONVENES  ⚖️              ║
                    ╚══════════════════════════════════════════════════════════╝

      ┌─────────────────┐      ┌─────────────────┐      ┌─────────────────┐
      │  ┌───────────┐  │      │  ┌───────────┐  │      │  ┌───────────┐  │
      │  │ ╭───────╮ │  │      │  │ ╭───────╮ │  │      │  │ ╭───────╮ │  │
      │  │ │GPT5.4│ │  │      │  │ │ CLAUDE│ │  │      │  │ │GEMINI │ │  │
      │  │ ╰───────╯ │  │      │  │ ╰───────╯ │  │      │  │ ╰───────╯ │  │
      │  │   ◉ ◉     │  │      │  │   ◉ ◉     │  │      │  │   ◉ ◉     │  │
      │  │    ⌣      │  │      │  │    ▽      │  │      │  │    ○      │  │
      │  └───────────┘  │      │  └───────────┘  │      │  └───────────┘  │
      │    JUDGE #1     │      │    JUDGE #2     │      │    JUDGE #3     │
      └────────┬────────┘      └────────┬────────┘      └────────┬────────┘
               │                        │                        │
               │ "I propose we use      │ "Actually, I must      │ "Interesting, but
               │  a vector database..." │  respectfully disagree" │  what about...?"
               │                        │                        │
               └────────────────────────┼────────────────────────┘
                                        ▼
                         ┌──────────────────────────────┐
                         │     🔥 ADVERSARIAL DEBATE 🔥   │
                         │                              │
                         │  GPT5.4: "Your approach has  │
                         │          a cold start issue" │
                         │                              │
                         │  CLAUDE: "Fair, but yours    │
                         │          doesn't scale"      │
                         │                              │
                         │  GEMINI: "Both valid. What   │
                         │          if we combine..."   │
                         └──────────────┬───────────────┘
                                        ▼
                         ┌──────────────────────────────┐
                         │      ✅ VERDICT REACHED ✅     │
                         │                              │
                         │   Synthesized best ideas     │
                         │   Schema-validated output    │
                         │   Confidence: 94%            │
                         └──────────────────────────────┘

[Council] Task completed in 45.2s | 3 judges | 2 debate rounds | Cost: $0.12

The LLM Council - Multiple AI models debating as judges

License: MIT Python 3.10+ OS: Cross-platform Code style: ruff Type checked: mypy

A multi-model orchestration package that runs adversarial council workflows across OpenAI, Anthropic, Google, Vertex, OpenRouter, and local CLIs.

This is not a Claude-only framework. Claude Code is one supported client and one supported provider path among several.

For tool-driven calls from Codex, Claude Code, Hermes or CI, use the portable invocation contract. It covers installed-binary identity, timeouts, process completion, credential boundaries, complete result files and execution-status handling. The shipped skill and caller examples require Council 0.8.4 or later for capability-based Claude admission and the earlier reporting fixes. See the 0.8.4 release notes for verification and limits.

Claude Code compatibility is capability-based, not an exact-build allowlist. Before each request, Council reads the selected binary's version for diagnostics and checks its help option definitions for the controls the request uses. New builds do not need a Council release when those controls remain available. Missing controls fail explicitly; Council never drops isolation flags to proceed. Help advertises syntax, not authentication, isolation semantics or generation success. Existing policy checks and strict terminal-response validation remain. Codex's allowlist is unchanged: 0.149.1, 0.153.3 and 0.160.1. Unknown Codex versions still fail explicitly. Report the chosen path/version from diagnostics, not an assumed shell binary; do not edit package constants to bypass a check. See the invocation contract for native versus synthetic verification boundaries. The Codex default remains gpt-5.4, which is absent from the tested 0.160.1 catalog. That version requires an explicit supported model through --models or provider configuration; contained native checks used gpt-6.1-sol. Exact 0.160.1 tool controls preserve authentication, selected model and reasoning effort. This is not support for every model or future CLI release.

This release also includes a mode-aware execution path with runtime profiles, routed handoff, capability planning, and deterministic eval tooling. Those capabilities materially extend the package runtime and make the public surface more explicit for planning, review, security, and research workflows.

The 0.8.0 release fixes Windows large-prompt handling in the CLI provider adapters (stdin instead of argv), surfaces Codex turn.failed server rejections instead of empty successes, restores Claude Code CLI auth on subscription and Vertex machines, and isolates CLI subprocess working directories from the caller's repo.

The 0.7.18 release adds provider-specific prompt-cache support. Cache request controls are adapter-owned, cache telemetry is normalized when providers return it, and Gemini/Vertex cached-content resources can be created, refreshed, and cleaned up with explicit lifecycle metadata.

Why Use a Council?

Single-model outputs have blind spots. By running multiple models in parallel and having them critique each other, the council:

  • Catches errors that any single model might miss
  • Reduces hallucination through cross-validation
  • Produces higher-quality outputs via adversarial refinement
  • Validates structure with JSON schema enforcement and retry logic

Features

Feature Description
Multi-Model Council Run Claude, GPT-5.4, and Gemini in parallel via OpenRouter or direct APIs
Mode-Aware Runtime drafter, critic, and planner honor runtime modes and execution profiles
Adversarial Critique Built-in critique phase identifies weaknesses and blind spots
Schema Validation JSON schema validation with automatic retry for structured outputs
Provider Agnostic Swap between OpenRouter, direct APIs, or CLI-based providers
Prompt Cache Support Anthropic/OpenRouter controls, OpenAI telemetry, and Gemini/Vertex cached-content lifecycle handling
Deep Doctor council doctor --deep checks real non-interactive generation readiness
Graceful Degradation Automatic retry, fallback, and skip strategies for failures
Artifact Store Persistent storage of drafts with tiered summarization
Eval Tooling eval, eval-compare, and local-only eval-import-pr support reproducible checks
Secret-Safe Logging Redaction pipeline prevents credential leakage

Requirements

Requirement Details
Python 3.10, 3.11, or 3.12
OS macOS, Linux, Windows (native or WSL)
Credentials At least one provider credential or authenticated local CLI (see below)

Supported Providers

Provider Environment Variable / Auth Notes
OpenRouter OPENROUTER_API_KEY Recommended - single key for all models
OpenAI OPENAI_API_KEY Direct OpenAI API access (GPT models)
Anthropic ANTHROPIC_API_KEY Direct Anthropic API access (Claude models)
Gemini API GOOGLE_API_KEY or GEMINI_API_KEY Direct Gemini API access
Vertex AI GOOGLE_CLOUD_PROJECT or ANTHROPIC_VERTEX_PROJECT_ID + ADC Enterprise GCP - Gemini + Claude
Claude Code Local claude CLI login CLI subprocess provider
Codex CLI Local codex CLI login CLI subprocess provider
Gemini CLI Local gemini CLI login CLI subprocess provider

Installation

pip install the-llm-council

With specific providers:

# OpenRouter (recommended - single API key for all models)
pip install the-llm-council

# Direct APIs
pip install the-llm-council[anthropic,openai,gemini]

# Vertex AI (Enterprise GCP)
pip install the-llm-council[vertex]

# All providers
pip install the-llm-council[all]

# Development
pip install the-llm-council[dev]

Agent Skills and Plugins

The LLM Council is available as an Agent Skill following the open Agent Skills standard. It works across OpenAI Codex, Claude Code, Cursor, VS Code, and other skill-compatible agents.

OpenAI Codex

# Copy skills directory to Codex skills location
cp -r skills/council ~/.codex/skills/

Claude Code

# Step 1: Add the repo as a marketplace
/plugin marketplace add sherifkozman/the-llm-council

# Step 2: Install the plugin
/plugin install llm-council@the-llm-council

Once installed, the council skill is auto-invoked when relevant, or use the /council command:

/council drafter --mode impl "Build a login page with OAuth"

Other Agents (Cursor, VS Code, GitHub, etc.)

Copy the skills/council/ directory to your agent's skills folder. The skill follows the open Agent Skills spec and works with any compatible agent.

Quick Start

CLI Usage

# Set your API key
export OPENROUTER_API_KEY="your-key"

# Run a council task (v0.7.x syntax with modes)
council run drafter --mode impl "Build a login page with OAuth"

# Multi-model council (Claude + GPT-5 + Gemini debating)
council run drafter --mode arch "Design a caching layer" \
  --models "anthropic/claude-opus-4-6,openai/gpt-5.4,google/gemini-3.1-pro-preview"

# Or set via environment variable
export COUNCIL_MODELS="anthropic/claude-opus-4-6,openai/gpt-5.4,google/gemini-3.1-pro-preview"
council run drafter "Build a login page"

# OpenRouter model IDs keep vendor namespaces like google/... .
# Those model IDs are separate from council provider names such as gemini or gemini-cli.

# Code review with security analysis
council run critic --mode review "Review auth changes" --verbose

# Ask the router to choose the next subagent/mode, then follow through
council run router "Assess whether we should add a hosted vector store" --route

# Bound latency/cost for review-style runs
council run critic --mode review "Review auth changes" \
  --runtime-profile bounded \
  --reasoning-profile off

# Disable artifact storage for faster runs
council run drafter "Quick fix" --no-artifacts

# Get structured JSON output
council run planner "Add user authentication" --json

# Legacy aliases still work, but prefer canonical subagents and modes
council run drafter --mode impl "Build a login page"

Python API

from llm_council import Council
from llm_council.protocol.types import CouncilConfig

# With mode configuration
config = CouncilConfig(providers=["openrouter"], mode="impl")
council = Council(config=config)
result = await council.run(
    task="Build a login page with OAuth",
    subagent="drafter"
)
print(result.output)

Check Provider Health

council doctor

# Verify actual non-interactive generation readiness
# May incur API/CLI usage.
council doctor --deep --provider claude --provider gemini-cli --provider codex

The Codex CLI adapter runs nested codex exec calls under an isolated temporary HOME, copying only the Codex auth files required for login. That keeps council subprocesses from inheriting the parent Codex agent's MCP tools, plugins, or skills while preserving your local Codex authentication. It also preserves the ambient Codex runtime environment, so council doctor reflects real login status instead of a false healthy result caused by over-stripped subprocess env. For Codex, a base doctor success reports login status only, not generation readiness. Require probe_ok: true from a deep probe for reachability; neither result proves large-review success. Do not infer path/version details absent from the diagnostic.

Run Deterministic Evals

# Public deterministic runtime checks
council eval evals/runtime-baseline.yaml --providers openrouter

# Compare named variants on the same dataset
council eval-compare evals/runtime-baseline.yaml variants.yaml --providers openai

For private benchmark creation, import external PR review material into .council-private/ only:

council eval-import-pr owner/repo 123

That path is gitignored and intended for local-only evaluation inputs. Do not commit imported diffs, copied code, or review fixtures into tracked repo paths.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                           LLM Council                               │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────────────────┐ │
│  │    CLI      │───▶│  Council    │───▶│     Orchestrator        │ │
│  │  (typer)    │    │   (API)     │    │                         │ │
│  └─────────────┘    └─────────────┘    │  ┌───────────────────┐  │ │
│                                        │  │  Health Checker   │  │ │
│  ┌─────────────────────────────────┐   │  ├───────────────────┤  │ │
│  │        Provider Registry        │◀──│  │ Degradation Policy│  │ │
│  │  ┌─────────┐ ┌─────────┐       │   │  ├───────────────────┤  │ │
│  │  │OpenRouter│ │Anthropic│ ...  │   │  │  Artifact Store   │  │ │
│  │  └─────────┘ └─────────┘       │   │  └───────────────────┘  │ │
│  └─────────────────────────────────┘   └─────────────────────────┘ │
│                                                                     │
│  ┌─────────────────────────────────────────────────────────────┐   │
│  │                    Subagent Configs                          │   │
│  │  router | planner | researcher | drafter | critic | ...     │   │
│  └─────────────────────────────────────────────────────────────┘   │
│                                                                     │
│  ┌─────────────────────────────────────────────────────────────┐   │
│  │                     JSON Schemas                             │   │
│  │  Validation & retry logic for structured outputs             │   │
│  └─────────────────────────────────────────────────────────────┘   │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

Pipeline Flow

0. HEALTH CHECK (optional)
   └── Preflight check of all providers, skip unhealthy ones

1. PARALLEL DRAFTS
   ├── Provider A generates draft
   ├── Provider B generates draft
   └── Provider C generates draft
   └── (Graceful degradation on failures)

2. ADVERSARIAL CRITIQUE
   └── Critic identifies weaknesses, contradictions, blind spots

3. SYNTHESIS
   └── Merge best elements, address critique, validate schema

4. VALIDATION
   └── JSON schema check with retry on failure

5. ARTIFACT STORAGE (optional)
   └── Store drafts and outputs for context management

Artifact storage defaults to ~/.council/ (artifacts/ plus ledger.db). You can override the root with COUNCIL_HOME, or set COUNCIL_ARTIFACT_DIR and COUNCIL_DB_PATH explicitly. Existing legacy ~/.claude/ council stores are still picked up automatically during migration.

Inspect or migrate storage explicitly:

council storage status
council storage migrate --dry-run
council storage migrate

Subagents (v0.7.x)

Core Agents

Subagent Modes Purpose Example
drafter impl, arch, test Generate code, designs, tests "Build the login page"
critic review, security Review and analyze "Review this PR for security"
synthesizer - Merge and finalize "Generate changelog for v1.2"
researcher - Technical research "Research OAuth providers"
planner plan, assess Roadmaps and decisions "Plan the auth implementation"
router - Classify and route tasks "Is this a bug or feature?"

Agent Modes

# drafter modes
council run drafter --mode impl "Build login page"     # Implementation (default)
council run drafter --mode arch "Design caching layer" # Architecture
council run drafter --mode test "Design test suite"    # Test design

# critic modes
council run critic --mode review "Review PR"           # Code review (default)
council run critic --mode security "Analyze auth"      # Security analysis

# planner modes
council run planner --mode plan "Plan implementation"  # Planning (default)
council run planner --mode assess "Redis vs Memcached" # Build vs buy

Legacy Aliases

Legacy names such as implementer, architect, reviewer, red-team, assessor, test-designer, and shipper still work for backwards compatibility, but public docs and examples now use the canonical subagents and modes.

Writing a Provider

Providers are pluggable via Python entry points. See the full Provider Development Guide for detailed instructions.

Quick Example

from llm_council.providers.base import ProviderAdapter, GenerateRequest, GenerateResponse

class MyProvider(ProviderAdapter):
    name = "myprovider"

    async def generate(self, request: GenerateRequest) -> GenerateResponse:
        # Your implementation
        return GenerateResponse(text="...", content="...")

    async def doctor(self) -> DoctorResult:
        return DoctorResult(ok=True, message="Healthy")

Register via pyproject.toml:

[project.entry-points."llm_council.providers"]
myprovider = "my_package.providers:MyProvider"

Reference Implementations

Provider Type File
OpenRouter HTTP API src/llm_council/providers/openrouter.py
Anthropic Native SDK src/llm_council/providers/anthropic.py
OpenAI Native SDK src/llm_council/providers/openai.py
Gemini API Native SDK src/llm_council/providers/gemini.py
Vertex AI Native SDK src/llm_council/providers/vertex.py
Claude Code CLI subprocess src/llm_council/providers/cli/claude_code.py
Codex CLI CLI subprocess src/llm_council/providers/cli/codex.py
Gemini CLI CLI subprocess src/llm_council/providers/cli/gemini_cli.py

Configuration

Environment Variables

# OpenRouter (recommended - single key for all models)
export OPENROUTER_API_KEY="your-key"

# Direct APIs
export ANTHROPIC_API_KEY="sk-ant-..."
export OPENAI_API_KEY="sk-..."
export GOOGLE_API_KEY="..."

# Vertex AI - Gemini (Enterprise GCP)
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_LOCATION="global"  # optional, default for Gemini on Vertex
export VERTEX_AI_MODEL="gemini-3.1-pro-preview"  # optional

# Vertex AI - Claude (Enterprise GCP)
export ANTHROPIC_VERTEX_PROJECT_ID="your-project-id"
export CLOUD_ML_REGION="global"              # Claude uses global region
export ANTHROPIC_MODEL="claude-opus-4-6@20260301"  # model with version

# Auth for Vertex AI: gcloud auth application-default login OR
# export GOOGLE_APPLICATION_CREDENTIALS="/path/to/sa.json"

# Multi-model council: comma-separated OpenRouter model IDs
export COUNCIL_MODELS="anthropic/claude-opus-4-6,openai/gpt-5.4,google/gemini-3.1-pro-preview"

# Per-provider model override (v0.7.0+)
export OPENAI_MODEL="gpt-5.4"               # Override OpenAI default
export ANTHROPIC_MODEL="claude-opus-4-6"     # Override Anthropic default
export GEMINI_MODEL="gemini-3.1-pro-preview" # Override Gemini API default
export OPENROUTER_MODEL="anthropic/claude-opus-4-6"  # Override OpenRouter default

# Model pack overrides for specific task types
export COUNCIL_MODEL_FAST="anthropic/claude-haiku-4-5"        # Quick tasks
export COUNCIL_MODEL_REASONING="anthropic/claude-opus-4-6"    # Deep analysis
export COUNCIL_MODEL_CODE="openai/gpt-5.4"                   # Code generation
export COUNCIL_MODEL_CRITIC="anthropic/claude-sonnet-4-6"     # Adversarial critique
export COUNCIL_MODEL_GROUNDED="google/gemini-3.1-pro-preview" # Research tasks
export COUNCIL_MODEL_CODE_COMPLEX="anthropic/claude-opus-4-6" # Complex refactoring

Per-Subagent Reasoning Configuration (v0.3.0+)

Subagents can be configured with provider preferences, model overrides, and extended reasoning/thinking budgets in their YAML configs:

# src/llm_council/subagents/critic.yaml (security mode)
name: critic
model_pack: harsh_critic

# Provider preferences
providers:
  preferred: [anthropic, openai]
  fallback: [openrouter]
  exclude: [gemini]

# Model overrides per provider
models:
  anthropic: claude-opus-4-6
  openai: o3-mini
  gemini: gemini-3-pro

# Extended reasoning/thinking configuration
reasoning:
  enabled: true
  effort: high           # OpenAI o-series: low/medium/high
  budget_tokens: 32768   # Anthropic: 1024-128000
  thinking_level: high   # Google Gemini 3.x: minimal/low/medium/high
Provider Parameter Values Description
OpenAI effort low/medium/high Reasoning effort for o-series models
Anthropic budget_tokens 1024-128000 Extended thinking token budget
Gemini API thinking_level minimal/low/medium/high Gemini 3.x thinking level

Default Reasoning Tiers (v0.4.0+)

All subagents have pre-configured reasoning defaults based on task complexity:

Tier Subagents Config Use Case
High drafter (arch), critic, planner effort: high, budget_tokens: 16384 Deep analysis, critical decisions
Medium drafter (impl), researcher effort: medium, budget_tokens: 8192 Balanced code/research tasks
Disabled router, synthesizer, drafter (test) enabled: false Fast tasks, no overhead

Config File

# ~/.config/llm-council/config.yaml
providers:
  - name: openrouter
    default_model: anthropic/claude-opus-4-6
  - name: openai
    default_model: gpt-5.4
  - name: gemini
    default_model: gemini-3.1-pro-preview

defaults:
  providers:
    - openrouter
  timeout: 120
  max_retries: 3
  summary_tier: actions
  output_format: json  # or "rich" (default) — useful for non-interactive clients

Provider default_model is forwarded to each provider's constructor, overriding the hardcoded default. Per-provider env vars (e.g. OPENAI_MODEL) take precedence over the config file, and the --models CLI flag overrides both.

CLI Reference

council run <subagent> "<task>"    # Run a council task
council doctor                      # Check provider health
council config                      # Show configuration
council version                     # Show installed version

# Options
--mode             Agent mode (impl/arch/test for drafter, review/security for critic, etc.)
--providers, -p    Comma-separated provider list
--models, -m       Comma-separated OpenRouter model IDs for multi-model council
--files, -f        File paths as context (repeatable or comma-separated; 50,000 chars/file, 200,000 total)
--context, --system  Additional system context/instructions
--timeout, -t      Per-attempt timeout in seconds (10-3600; default 120)
--temperature      Model temperature (0.0-2.0)
--max-tokens       Max output tokens
--input, -i        Read task from file (use '-' for stdin)
--output, -o       Write output to file
--schema           Custom output schema JSON file
--dry-run          Show what would run without executing
--no-artifacts     Disable artifact storage
--json             Output structured JSON
--verbose, -v      Verbose output

File Context

Pass source files directly to council for review, implementation, or analysis:

# Comma-separated
council run critic --mode review --files src/auth.py,src/middleware.py "Review these files"

# Repeatable -f (cleaner for many files)
council run critic --mode review \
  -f src/auth.py \
  -f src/middleware.py \
  -f src/handler.py \
  "Review these files"

# Security audit
council run critic --mode security -f src/payment.py "Audit payment handler"

Limits are 50,000 characters per file and 200,000 retained characters in total, not KB or UTF-8 bytes. A file above the per-file limit is truncated with a warning. If the retained total exceeds the total limit, the run fails before provider calls; it does not silently skip the excess file.

Inspect execution_plan.context_preparation.files for path, original_chars, retained_chars, and truncated. Truncation is also reported in degradation_report.context_warnings with kind: "file_truncated", even when graceful degradation is disabled. A successful run with truncated input has execution_status: "degraded". Do not describe it as a full-file review. Review the omitted material separately or narrow the input and state the actual scope.

Execution And Evidence Reporting

Use execution_status, not success alone, in JSON, Markdown and console output. A valid degraded result can have success: true and exit 0. A completed result does not prove substantive independent reviews or correct findings. Inspect drafts, provider_errors, degradation_report, execution_plan and run_id.

For the built-in reviewer, critique and synthesis receive bounded prepared-coverage metadata as data, not source evidence or proof of every-phase delivery. The review prompt separates Council pipeline limitations from source findings; real defects in pipeline code under review still require source evidence. This added prompt boundary does not apply to security or custom schemas and adds no task-intent detector or semantic regex filter. Public result schemas are unchanged; the adapter model-resolution hook described below is additive.

Schema-validation exhaustion returns failed, preserving drafts, critique and validation errors. Council no longer manufactures reviewer findings from arbitrary draft/critique prose by inferring severity, category, location or remediation. Invalid raw synthesis is retained only as an artifact when storage succeeds, not promoted to valid output. A synthesis exception can still use an existing schema-valid JSON draft as a degraded fallback. Schema validity is not semantic truth; inspect the findings and diagnostics rather than treating fallback as approval.

File ingestion is not delivered coverage. The context_preparation.files counts above describe files read, before source selection and downstream prompt compaction. Inspect execution_plan.context_preparation.coverage and execution_plan.phase_prompt_compaction for delivered source and draft evidence. Even profile index 0 can cut content. truncated: false at ingestion does not establish full delivery to each phase. Council slicing, cut draft text and schema omission are pipeline limitations, not defects in the source under review or proof that a provider generated incomplete text. Compaction metadata includes original_prompt_chars and delivered_prompt_chars; evidence_compacted excludes schema-only omission. submitted distinguishes unsent candidates from adapter dispatch; only submitted evidence cuts count toward compaction-based degradation.

degradation_report.total_retries counts extra calls actually started; planned_retries separately records decisions, not executed work or retry budgets. Inspect synthesis_attempts even when the final output comes from a draft fallback; it counts actual adapter starts, not queued requests that never start. An empty fallbacks_used list alone cannot rule out final-output fallback. Handled cancellation retains completed evidence where available, not unfinished work. Inspect execution_plan.artifact_occurrences for each phase/provider's artifact reference and execution_plan.persistence for storage/finalization errors. Identical text does not make two phase occurrences the same evidence. Disabled persistence is not a storage failure; absent artifacts are not proof that nothing executed.

With persistence enabled, TOOL_LOG manifests of type council_execution capture started and settled snapshots of allowlisted runtime, attempt, occurrence, coverage and persistence-error metadata. The settled snapshot records ledger_finalization: not_yet_attempted; read the terminal ledger separately. A hard kill may retain only the start snapshot; learned native identity is not durable in these manifests until settlement is persisted. No heartbeat or automatic repair is added.

A hard kill can leave a ledger row marked running. That row does not prove process liveness, and a stopped caller does not prove every descendant exited. Check the original process handle and caller termination evidence. Do not automatically repair ledger rows or restart paid runs.

execution_plan.runtime_identity identifies the Council entrypoint, imported package and Python runtime for CLI and library callers through a shared engine helper. Native diagnostics must distinguish the selected executable and observed version from the requested model. A requested model is not a provider-confirmed model identity. Inspect execution_plan.provider_attempts for cli_path, cli_realpath, cli_version, requested_model, adapter_resolved_model and adapter_reported_model where available. The resolved model can come from the adapter's default, not an explicit caller choice; neither resolution nor adapter reporting is independent provider confirmation. Do not infer historical executable selection from the current shell, or fill in identity fields absent from a report.

Before applying capability policy, the engine resolves an omitted model from the instantiated adapter's default. The optional, non-abstract resolve_model hook is backward compatible: it does not re-read environment variables, change defaults or override an explicit model; unknown defaults remain unknown. This restores schema forwarding for the OpenAI adapter's gpt-5.4 default. It does not restore or enable gpt-5.4 reasoning controls: Council's existing OpenAI capability policy forwards those controls only for o-series models. That policy and model defaults are unchanged. Do not claim that high reasoning reached gpt-5.4; inspect request-compilation metadata. O-series reasoning forwarding is covered separately by regression checks.

--models takes literal model IDs in provider order, not provider-prefixed provider:model mappings: codex:gpt-5.4 and vertex-ai:gemini-3.1-pro-preview are invalid mappings. Preserve official IDs such as openai/gpt-5.4 and qwen/qwen3.6-plus:free for providers that accept them; slash namespaces and colon suffixes are not a reason to rewrite a model ID.

Updating A Pinned uv Tool

After publication and approval to update the installed tool, preserve any local package edits and the uv receipt, then replace the exact version constraint:

uv tool install --reinstall 'the-llm-council[all]==0.8.4'

This targets the uv-managed tool and its exposed executable, and updates the stored pin. Installing into an isolated test environment does not update the command used by real callers. uv tool upgrade respects the existing constraint; an exact ==0.8.2 pin must be replaced explicitly. See uv tool upgrade rules. Verify the resolved Council path, version, imported source and receipt from each actual caller. Report native CLI selection mismatches without deleting native installations or changing PATH. This instruction does not itself authorize an installation, prove publication, or establish live-provider compatibility.

Timeout Budgets

--timeout accepts 10 through 3600 seconds; its default remains 120. It limits each provider attempt, not the whole workflow. Retries, multiple phases, queue waits, backoff and cleanup can make the workflow longer. Set the caller's outer deadline separately and wait for the original process to finish before reading its result or considering another run.

For long-deadline work, use --runtime-profile default. The bounded profile keeps its shorter provider/phase caps even with --timeout 3600. Inspect execution_plan.provider_request_timeouts_by_provider for effective budgets. A successful doctor --deep probe proves reachability only, not large-review completion, full input coverage, or review quality.

Development

# Clone the repository
git clone https://github.com/sherifkozman/the-llm-council.git
cd the-llm-council

# Install with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Run linting
ruff check src/
mypy src/llm_council

Contributing

Contributions are welcome! See our Roadmap for planned features and Contributing Guide for details.

Quick Start

# Fork and clone
git clone https://github.com/YOUR_USERNAME/the-llm-council.git
cd the-llm-council

# Install dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Run linting
ruff check src/ && mypy src/llm_council

Contribution Workflow

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Make your changes
  4. Test your changes (pytest)
  5. Lint your code (ruff check src/ && mypy src/llm_council)
  6. Commit with a clear message (git commit -m 'Add amazing feature')
  7. Push to your branch (git push origin feature/amazing-feature)
  8. Open a Pull Request

What We're Looking For

  • New Providers: Add support for more LLM backends
  • New Subagents: Create specialized agents for specific tasks
  • Bug Fixes: Found a bug? We'd love a fix!
  • Documentation: Improvements to docs are always welcome
  • Tests: More test coverage is great

Security

For security concerns, please see our Security Policy or email vibecode@sherifkozman.com.

Key security features:

  • CLI adapters use exec-style subprocess (no shell injection)
  • Environment variable allowlisting prevents secret leakage
  • Path traversal protection in artifact storage
  • Configurable secret redaction in logs

License

MIT License - see LICENSE for details.

Acknowledgments

Built with:


When one model isn't enough, convene a council.

~ vibe coded by Sherif Kozman & The LLM Council ~

Metadata

Release files for the-llm-council 0.8.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for the-llm-council 0.8.4
File Size Uploaded
the_llm_council-0.8.4.tar.gz 364.6 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for the-llm-council 0.8.4
File Interpreter ABI Platform
the_llm_council-0.8.4-py3-none-any.whl Python 3 none any Details

Total release size: 622.4 kB

Release files / the_llm_council-0.8.4.tar.gz

Download URL the_llm_council-0.8.4.tar.gz
Size 364.6 kB
Tags Source
SHA-256 checksum
How to use checksums
3e89b3a0ea4fab700dc63eabd847fa4658c2253e7137c0f3c74cb982e741189a
BLAKE2b-256 checksum
How to use checksums
046b62462be56c38313d575f9ae0ee1605d14b943a0d2257b2770a11d9053637
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 9, 2026.

Transparency log

Release files / the_llm_council-0.8.4-py3-none-any.whl

Download URL the_llm_council-0.8.4-py3-none-any.whl
Size 257.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
60487b512038ec99c80681d5549cfcb486f69f5c77c6140d129faede4e430223
BLAKE2b-256 checksum
How to use checksums
49cca6de58e7167b238a954701faa5cfe0c41298e7817bc5e12a2f23da06c504
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 9, 2026.

Transparency log

Release history Release notifications | RSS feed

This release

0.8.4 This release

2 release files

0.8.3

2 release files

0.8.2

2 release files

0.8.1

2 release files

0.8.0

2 release files

0.7.18

2 release files

0.7.16

2 release files

0.7.9

2 release files

0.7.8

2 release files

0.7.7

2 release files

0.7.6

2 release files

0.7.5

2 release files

0.7.4

2 release files

0.7.3

2 release files

0.7.2

2 release files

0.7.1

2 release files

0.7.0

2 release files

0.6.0

2 release files

0.5.3

2 release files

0.5.2

2 release files

0.5.1

2 release files

0.5.0

2 release files

0.4.13

2 release files

0.4.12

2 release files

0.4.11

2 release files

0.4.10

2 release files

0.4.9

2 release files

0.4.8

2 release files

0.4.7

2 release files

0.4.6

2 release files

0.4.5

2 release files

0.4.4

2 release files

0.4.3

2 release files

0.4.2

2 release files

0.4.1

2 release files

0.4.0

2 release files

0.3.1

2 release files

0.3.0

2 release files

0.2.3

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page