Skip to main content

A TDD-Driven, Parallelized AI Coding Orchestrator

Project description

Shard

A TDD-driven, parallelized AI coding orchestrator that breaks down complex tasks and executes them concurrently using git worktrees.

What is Shard?

Shard takes a natural language prompt describing a coding task and automatically:

  1. Plans - Uses an LLM to decompose your task into a DAG of parallel sub-tasks, each with exclusive file ownership
  2. Partitions - Creates isolated git worktrees so agents can work simultaneously without conflicts
  3. Dispatches - Runs AI coding agents (Claude Code, Aider, or Cursor) in parallel across tasks
  4. Aggregates - Merges all branches back together, resolving structural conflicts automatically
  5. Self-heals - Runs your test suite and automatically fixes failures

Installation

Prerequisites

Required:

  • Python 3.11+
  • Git 2.20+

AI Agent (install one):

# Claude Code (recommended)
npm install -g @anthropic-ai/claude-code

# Aider
pip install aider-chat

# Cursor CLI
# Install from https://cursor.sh

Install Shard

# From PyPI
pip install shard-code

# From source
git clone https://github.com/nihalgunu/Shard.git
cd Shard
pip install -e ".[dev]"

Quick Start

cd your-project

# Run a full pipeline
shard run -p "Add user authentication with JWT tokens and refresh token support"

# Preview the execution plan without running
shard plan -p "Refactor the database layer to use async operations"

# Check status of current run
shard status

# Resume an interrupted run
shard resume <run-id>

# View logs for a specific task
shard logs <task-id>

Commands

Command Description
shard run -p "..." Execute a full pipeline with the given prompt
shard run -f prompt.txt Execute using prompt from file
shard plan -p "..." Preview execution DAG without running
shard status Show current execution status
shard resume <run-id> Resume an interrupted run
shard logs <task-id> View agent stdout/stderr
shard abort <run-id> Stop all running agents
shard clean <run-id> Remove worktrees, branches, artifacts
shard config Show current configuration

Run Options

shard run -p "Your prompt" \
  --agents 4 \              # Max concurrent agents (default: 4)
  --backend claude-code \   # Agent backend: claude-code, aider, cursor-cli
  --timeout 3600 \          # Global timeout in seconds
  --max-cost 10.0           # Maximum spend in USD

Configuration

Create shard.toml in your repository root:

[agent]
backend = "claude-code"     # claude-code | aider | cursor-cli | custom
binary = ""                 # Custom agent binary path (for custom backend)
max_concurrent = 4          # Max parallel agents
stagger_delay_s = 2.0       # Delay between agent launches

[planner]
provider = "anthropic"      # anthropic | openai
model = "claude-sonnet-4-20250514"  # or "gpt-4o" for OpenAI
temperature = 0.2

[timeouts]
per_task_s = 600            # Per-task timeout
global_s = 3600             # Global timeout
output_stall_s = 120        # Kill agent if no output for this long
commit_stall_s = 300        # Kill agent if no commit for this long

[retries]
max_per_task = 3            # Max retries per failed task
max_global = 5              # Max total retries across all tasks

[cost]
max_usd = 5.00              # Hard spending limit
warn_usd = 3.00             # Warning threshold

[git]
worktree_dir = "../worktrees"  # Where to create worktrees
branch_prefix = "wt"           # Branch naming prefix
auto_cleanup = true            # Clean up on success

[test]
runner = "pytest"           # Test command
args = ["-xvs"]             # Test arguments
json_report = true          # Use pytest-json-report for better parsing

[logging]
level = "INFO"              # DEBUG | INFO | WARNING | ERROR
format = "json"             # json | text

Environment Variables

Set the API key for your chosen planner provider:

# For Anthropic (default)
export ANTHROPIC_API_KEY=sk-ant-...

# For OpenAI
export OPENAI_API_KEY=sk-...

Why Git Worktrees?

Traditional approaches to parallel AI coding have a problem: file conflicts. If two agents try to edit the same file simultaneously, you get race conditions, merge conflicts, or corrupted state.

Shard solves this with git worktrees:

Approach Problem
Single working directory Agents overwrite each other's changes
Multiple clones Wastes disk space, slow to set up
File locking Serializes work, kills parallelism
Git worktrees Lightweight, isolated, native git support

How worktrees help:

  • Each agent gets its own complete working directory
  • All worktrees share the same .git folder (minimal disk overhead)
  • Each worktree is on its own branch
  • Merging is just git merge - git handles the complexity
  • Native git tooling works everywhere

This means 4 agents can simultaneously edit 4 different parts of your codebase with zero coordination overhead.

How It Works

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         SHARD PIPELINE                          │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────┐  │
│  │  Stage 1 │───▶│  Stage 2 │───▶│  Stage 3 │───▶│  Stage 4 │  │
│  │ Planner  │    │Partitioner│   │Dispatcher │   │Aggregator│  │
│  └──────────┘    └──────────┘    └──────────┘    └──────────┘  │
│       │                               │               │         │
│       ▼                               ▼               ▼         │
│   DAG + Tests              Parallel Agents      Merge + Test    │
│                                   │                   │         │
│                                   │          ┌───────┴───────┐  │
│                                   │          │    Stage 5    │  │
│                                   │          │  Self-Healer  │  │
│                                   │          └───────────────┘  │
└─────────────────────────────────────────────────────────────────┘

Stage Details

Stage 1: Planner

  • Analyzes your codebase structure
  • Decomposes prompt into independent tasks
  • Assigns exclusive file ownership to prevent conflicts
  • Generates test scaffolds (TDD approach)

Stage 2: Partitioner

  • Creates a git worktree for each task
  • Each worktree branches from a common scaffold commit
  • Enables true parallel execution

Stage 3: Dispatcher

  • Launches AI agents respecting DAG dependencies
  • Monitors output, cost, and timeouts
  • Live TUI shows progress across all agents

Stage 4: Aggregator

  • Merges completed branches in topological order
  • Auto-resolves structural conflicts (package.json, requirements.txt, etc.)
  • Runs test suite on merged result

Stage 5: Self-Healer

  • Maps test failures back to responsible tasks
  • Re-dispatches agents with failure context
  • Bounded retry loop prevents infinite loops

State & Artifacts

Shard stores state in .shard/ in your repo:

.shard/
├── graph.json          # Current execution DAG
├── journal/            # Write-ahead event log
├── prompts/            # Per-task prompt files
├── logs/               # Agent stdout/stderr
└── checkpoints/        # Binary snapshots for fast resume

Worktrees are created in ../worktrees/<run-id>/ by default.

Examples

Add a Feature

shard run -p "Add a REST API endpoint for user profiles with GET, POST, PUT, DELETE operations. Include input validation and proper error handling."

Refactor Code

shard run -p "Refactor the authentication module to use dependency injection. Update all tests accordingly."

Fix a Bug

shard run -p "Fix the race condition in the connection pool that causes intermittent timeouts under high load."

Preview Only

shard plan -p "Implement caching layer for database queries"
# Shows DAG without executing - review before committing resources

License

Apache 2.0

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

shard_code-1.0.2.tar.gz (41.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

shard_code-1.0.2-py3-none-any.whl (41.8 kB view details)

Uploaded Python 3

File details

Details for the file shard_code-1.0.2.tar.gz.

File metadata

  • Download URL: shard_code-1.0.2.tar.gz
  • Upload date:
  • Size: 41.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.12

File hashes

Hashes for shard_code-1.0.2.tar.gz
Algorithm Hash digest
SHA256 f61818576d07621a14b0dd2354cd4f4faadef3447b293d0bdb9f6e6f79cdabf3
MD5 6c76c8acb3c5afef068be2b0d4f00781
BLAKE2b-256 3c650534f4c1dea2f1f3b76d3a101890a26b0426f6f08112f9d81319e6c434f9

See more details on using hashes here.

File details

Details for the file shard_code-1.0.2-py3-none-any.whl.

File metadata

  • Download URL: shard_code-1.0.2-py3-none-any.whl
  • Upload date:
  • Size: 41.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.12

File hashes

Hashes for shard_code-1.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 4b6681462eeddae5e4bdc37c9ffcb6e34693aa5a74db67c60784bdab36c4e056
MD5 61619ed00bf94da1e25d946b59f67df3
BLAKE2b-256 33f26d6078e69a0a8dc0a2918c22fefcc11f4b568cc50378da58dc2aab020027

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page