🤖 CoderAI

Autonomous AI Pair Programming & Multi-Agent Swarm in Your Terminal

Overview • Benchmarks • Quickstart • Interactive CLI • Agent Roles & Swarms • Core Tools • Security • Configuration • Development • License

Overview

CoderAI is an autonomous terminal AI software engineering platform, agentic execution engine, and multi-agent swarm designed for high reliability, deterministic tool execution, token efficiency, and developer velocity. It couples a session runtime (coderai.soul.session.manager), a terminal interface (coderai.ui.shell), and a hybrid Kahneman System 1 (Jev / TypeSafe) + System 2 (CoderAI Deep Review) compound reasoning harness for ultra-fast, high-precision code review.

See Runtime architecture for execution boundaries and Verification for the shared checks and release gates.

• 🎯 Snippet-Anchored Editing: Precise, hash-anchored edits with real-time staleness detection to eliminate hallucinated overwrites.
• 🔄 Bounded Agent Loop: Deterministic turn execution with loop guards, repeat reminders, and token compaction.
• ⚡ JEV System-One AI Compound Engine: Sub-second non-autoregressive diff screening & precision gating via TypeSafe, shielding developers from false-positive hallucinations and cutting review turnaround to under 7 seconds.
• 🛡️ Defense-in-Depth Security: 10 fine-grained permission scopes, 3 security presets, and native OS sandboxing (Seatbelt / Bubblewrap).
• ⏪ Turn Checkpoints & Instant Undo: Automatic .git_history snapshots enable one-command rollback (/undo) and diff inspection (/diff).
• 🔍 High-Performance Search: Bundled native ripgrep binary with spill-to-disk locators for large search outputs.
• 🤖 Multi-Agent Teams & Swarm: Continuable background subagents, shared team task boards, and automated verification loops (ralph).
• 🧠 Frontier Models & MCP: Native reasoning token support (OpenAI, DeepSeek, Gemini, Claude) and Model Context Protocol integration.

🏆 Martian Code Review Benchmark Results

CoderAI was independently evaluated on the Martian Code Review Benchmark across 50 real-world pull requests across 5 production enterprise repositories (cal.com, keycloak, grafana, sentry, discourse), competing against 30 leading commercial and open-source AI code review tools.

CoderAI placed Rank 5 in the World, achieving top-tier precision and balanced recall while outperforming general-purpose generative models and established commercial review bots:

Key Benchmark Takeaways

• Eliminating the "Noise Problem": Conventional generative code review agents produce 130 to 260+ False Positives across 50 PRs (CodeRabbit: 235 FPs / 30.7% Precision, Copilot: 201 FPs / 35.2% Precision, Cubic-dev: 266 FPs / 27.3% Precision), causing severe developer alert fatigue. CoderAI produced only 56 False Positives (1.1 per PR) with 58.2% Precision.
• The Jev Precision Lift: The Kahneman System 1 Precision Gate (coderai/triage/) blocked 38 speculative/hallucinated findings, yielding a +13.0% Precision gain and lifting CoderAI from Rank 12 into the Global Top 5.
• Sub-7 Second Turnaround: Total review turnaround averaged 6.63 seconds per PR (median 5.29s), delivering instant feedback compared to 2–8 minute latencies for commercial bots.
• Repository Performance: CoderAI achieved 64.9% F1 on cal.com (69.4% Precision), 58.8% F1 on keycloak (65.2% Precision), and 55.3% F1 on grafana (59.1% Precision).
• For complete per-repository statistics and profile scoring, see docs/benchmarks.md.

Installation

From Source (Editable Mode)

# Clone the repository
git clone https://github.com/adityaanilraut/CoderAI.git
cd CoderAI

# Install in editable mode
pip install -e .

# Verify installation
coderai --version
# or using shorthand alias
cai --version

Install with Development Dependencies

.venv/bin/python -m pip install -e ".[dev,jev]" -e sdks/coderai-sdk

Quickstart

1. Configure Providers, API Keys & Models (Interactive Setup)

You can run the interactive setup wizard directly from the command line or from within CoderAI:

# Interactive setup wizard
coderai setup

# Or configure non-interactively
coderai setup --provider deepseek --key sk-... --setup-model deepseek-v4-pro
coderai setup --provider gemini --key AIzaSy... --setup-model gemini-3.7-flash
coderai setup --base-url http://localhost:11434/v1 --setup-model qwen2.5-coder:32b --provider ollama

# Test active provider connectivity
coderai setup --test

# View configured keys & endpoints status table
coderai setup --status

Alternatively, you can export environment variables in your shell or .env file:

export OPENAI_API_KEY="sk-..."
export DEEPSEEK_API_KEY="sk-..."
export GEMINI_API_KEY="AIzaSy..."
export ANTHROPIC_API_KEY="sk-ant-..."

2. Launch Interactive REPL

coderai
# or
cai

Inside the REPL, you can type /setup at any time to add or update API keys, switch default models, or configure local LLM endpoints.

3. Launch in Plan Mode

coderai --plan

4. Run a One-Shot Task

coderai "Refactor auth middleware to support JWT refresh tokens"

One-shot prompts default to the core tool preset. Use --preset to select one of:

• full: all built-in tools
• core: shell, file editing, reads, writes, glob, and grep
• shell_edit: only bash and str_replace_editor

5. Auto-Approve Permissions for CI / Scripting

coderai --yes "Run unit tests and fix any failing assertions in tests/test_core.py"

6. Launch with Specialized Agent Roles

# Launch interactive session with a specific role
coderai --agent architect
coderai --agent tdd-guide

# Launch with custom role specification file
coderai --agent-file .coderai/agents/code-reviewer.md

Interactive CLI & Slash Commands

When running coderai, you enter an interactive REPL featuring an ASCII banner, real-time token status bar, thinking block display, live diff rendering, and command autocompletion.

Slash Commands

Tab Autocompletion & Command History

CoderAI includes full readline and fuzzy autocompletion:

• Tab autocompletion: Press Tab on / commands, /model targets, and @ file paths.
• Persistent history: Arrow Up / Down to cycle through previous prompts across sessions (persisted at ~/.coderai/history).

Context Expansion (@file Mentions)

Mention workspace files directly anywhere in your prompts:

coderai> Explain the architecture in @coderai/soul/session/manager.py and how it connects to @coderai/soul/approval.py

CoderAI automatically detects referenced files and attaches their contents to the prompt context. Specific line ranges can also be targeted with @file.py:10-30 or @file.py:L25.

Agent Roles & Swarms

CoderAI supports dynamic discovery of specialized markdown agent specifications (.coderai/agents/*.md, .agents/agents/*.md, ~/.agents/agents/*.md), as well as decentralized multi-agent swarms:

Swarm Coordination

• spawn_teammate: Decentralized agent spawning with isolated inboxes and priority actor channels.
• TeamTaskBoard: DAG-validated task boards preventing circular dependencies and tracking blocked/in-progress statuses.
• wait_agent: Synchronization barriers supporting completion, message settlement, or timeout-based join.

JEV System-One AI Model & Compound Review Architecture

CoderAI incorporates a hybrid Kahneman System 1 + System 2 compound review harness (coderai/triage/, coderai/jev/). While generative autoregressive models (System 2) excel at deep syntactic, semantic, and concurrency reasoning, they suffer from verbosity and uncalibrated confidence—frequently hallucinating speculative warnings or pedantic nitpicks that cause developer fatigue.

To resolve this, CoderAI pairs System 2 with Jev (jev-system-one via TypeSafe), a specialized non-autoregressive System 1 model providing sub-second diff screening and calibrated probability gating:

┌─────────────────────────────────────────────────────────────────────────────┐
│  Tier 1: System 1 Diff Triage (Jev screen_diff_hunk)                        │
│  • Sub-100ms screening of git diff hunks                                    │
│  • Bypasses inert files (lockfiles, assets, build metadata)                 │
│  • Flags high-risk hunks and concentrates System 2 attention                │
├─────────────────────────────────────────────────────────────────────────────┤
│  Tier 2: System 2 Deep Reasoning Review (CoderAI + Frontier LLM)            │
│  • Multi-file AST, semantic, and concurrency analysis                        │
│  • Catches API contract violations, dead allocations, and race conditions   │
│  • Drafts concrete findings with file/line references and failure mechanisms│
├─────────────────────────────────────────────────────────────────────────────┤
│  Tier 3: Pre-Flight Precision Gate (Jev gate_candidate_comment)             │
│  • Non-autoregressive calibrated confidence verification                    │
│  • Questions: is_actionable_bug, is_speculative_or_nit, will_developer_accept│
│  • Blocks ungrounded hallucinations, hypothetical edge cases, and noise     │
│  • Delivers a +13% Precision lift (58.2% Precision vs 30-35% for peer bots) │
└─────────────────────────────────────────────────────────────────────────────┘

Core Capabilities & Implementation Details:

• Non-Autoregressive Fast Inference: Single-shot system_one(state, questions) execution without streaming or tool overhead for instantaneous code triage (~0.5s).
• Calibrated Tunables: Configurable via environment variables or settings:
  • CODERAI_JEV_GATE_THRESHOLD (default 0.40–0.50): Minimum is_actionable_bug probability required for approval.
  • CODERAI_JEV_SPECULATIVE_THRESHOLD (default 0.65): Maximum ceiling for speculative/nit warnings before rejection.
  • CODERAI_JEV_ACCEPT_THRESHOLD (default 0.50): Minimum predicted senior developer acceptance probability.
• Fail-Safe Fallback: Guaranteed graceful degradation to standard System 2 execution upon timeout or network latency.
• LRU In-Memory Caching: Bounded thread-safe caching (CODERAI_JEV_CACHE_SIZE) with hit/miss telemetry and hash-keyed lookups.
• Setup: pip install 'coderai-agent[jev]' and set TYPESAFE_API_KEY (env, .env, or /setup). Without either, /review operates in standalone System 2 mode.
• Data egress & Security: When Jev is active, /review sends each changed file's diff (up to CODERAI_JEV_MAX_DIFF_CHARS) and each drafted comment to the TypeSafe API, separately from your chat model provider. Secret-looking paths (.env*, *.pem, *.key, *credential*, *secret*, SSH keys) are never sent; they are always reviewed and their comments always shown.
• Read more in docs/jev-system-one.md.

Core Tools

CoderAI provides a rich, versatile tool surface:

Security & Permissions

CoderAI follows a defense-in-depth, fail-closed security model.

Permission Scopes

Permission Presets

• read-only: Blocks all mutations and confines execution to read operations with an active OS sandbox.
• workspace-write: Grants write access only within the workspace with an active OS sandbox.
• danger-full-access: Full access with user confirmation.

Configuration

Settings are resolved in order of precedence: CLI arguments > Environment variables > Project config (.coderai/settings.json) > User config (~/.coderai/settings.json).

Configuration Example (settings.json)

{
  "model": "gpt-6-luna",
  "temperature": 0.2,
  "thinkingEnabled": true,
  "reasoningEffort": "max",
  "toolsPreset": "core",
  "permissions": {
    "preset": "workspace-write",
    "allow": ["read-in-cwd", "query-git-log"],
    "ask": [
      "write-in-cwd",
      "write-out-cwd",
      "delete-in-cwd",
      "mutate-git-log",
      "network",
      "mcp"
    ],
    "deny": ["delete-out-cwd"]
  },
  "mcpServers": {
    "sqlite": {
      "command": "uvx",
      "args": ["mcp-server-sqlite", "--db-path", "./app.db"]
    },
    "remote-docs": {
      "url": "https://mcp.example.com/sse",
      "transport": "sse",
      "headers": {
        "Authorization": "Bearer token"
      }
    }
  }
}

Environment Variables

Migration Notes

• Project configuration and tracked agent content now use lowercase .coderai; rename .coderAI directories before upgrading.
• Workspace skills use the singular filename SKILL.md; rename legacy SKILLS.md files.
• Use --prompt (or a positional prompt) instead of the removed --message flag.
• --preset, --tools-preset, and --permission are aliases for the same tool-preset slot; pass only one.
• Tool presets accept only full, core, or shell_edit.
• Settings JSON uses the documented camelCase keys (baseURL, apiKey, thinkingEnabled, reasoningEffort, and toolsPreset); environment overrides use CODERAI_*.

Development

# Format code
make format

# Run linter
make lint

# Run type checks
make typecheck

# Run main, offline SDK and wire suites (one process per file)
make test

# Full local/CI verification, including the resolved dependency audit
make check

# Run offline engine self-check
.venv/bin/python scripts/self_check.py

# Clean build artifacts
make clean

Development commands use .venv/bin/python by default. The test-suite audit documents the completed local audit and retained contracts. See the verification contract for check scopes, required Jev/SDK coverage, tool pins and audit policy.

Acknowledgements

CoderAI builds upon and adapts architectural concepts, protocol designs, and patterns from pioneering open-source agentic projects:

• DeepSeek Harness (MIT License):

  • Snippet-Anchored Precision Editing: The robust read (anchored snippet_id) to edit pattern with whitespace tolerance and staleness detection.
  • Bounded Agent Loop & Token Compaction: Deterministic turn cycles, multi-turn loop guards, and intelligent context compaction.
  • Side-Effect Permission Scopes: The 10-scope fail-closed security architecture (read-in-cwd, mutate-git-log, mcp, etc.).
  • Interactive Terminal Workflow: Advanced input buffering, dynamic status bar, and interactive menu patterns.

• Kimi Code CLI (by Moonshot AI, MIT License):

  • Core Soul Runtime & Turn Execution: The multi-step turn/step life cycle controller, asynchronous tool execution loop, and JSONL turn context persistence.
  • Terminal UX & Slash Dispatch: Robust interactive REPL mechanics, modular slash command action dispatcher (/context, /clear, /sessions, /model), and dynamic prompt rendering.
  • Agent Toolset & Protocol Architecture: Built-in file manipulation tools (read, write, replace, grep, glob), media handling, and Agent Control Protocol (ACP) integration.

We gratefully acknowledge and thank the creators and contributors of both DeepSeek Harness and Kimi CLI for their foundational contributions to terminal-first agentic software engineering. In compliance with the MIT License terms, original copyright notices and attribution are maintained across adapted core modules.

License

MIT License. See LICENSE for details.