ICLayout-Bench

A benchmark framework for AI-driven integrated-circuit layout.
_{From netlists to GDS, with isolated EDA tools and independent evaluation.}

简体中文 · Quick Start · Documentation · Issues

Why ICLayout-Bench?

Measure how well an Agent turns a circuit, constraints and process resources into an integrated-circuit layout. Run experiments, compare agent configurations or connect your controller to an HTTP evaluation service.

Prerequisites

• Python 3.12+.
• An installed, authenticated agent CLI or your own controller.
• A task Dataset, such as the public ICLayout-Bench Dataset, supplied by repository ID or directory.
• For local evaluation: Linux x86-64, Docker and a compatible EDA image. Remote participation uses the operator's tools instead.

Quick Start

1. Install Bench.

python -m pip install iclayout-bench

For local wheel or source installation, see installation options. To run EDA on your machine, pull the published tools image:

docker pull docker.io/lizhangmai/iclayout-bench-tools:latest

You can also build or select another compatible image.

2. Write experiment.toml. Replace the model ID and effort with your settings. This example uses the public Dataset; source also accepts other Dataset IDs or directories. Configuration and output paths are yours to choose.

harness = "codex"
model = "your-model-id"
effort = "medium"
concurrency = 1
repetitions = 1

[dataset]
source = "MAI716/ICLayout-Bench"
name = "core"
split = "test"

3. Inspect one task, then run it. Bench downloads the selected Dataset on first use. This example selects NAND2_X1 from its core tasks.

# Inspect configuration and task bindings without model or Docker calls.
python -m benchmarking.run \
  --config experiment.toml --case NAND2_X1 --dry-run

# Run the Agent and independently evaluate its submitted layout.
python -m benchmarking.run \
  --config experiment.toml --case NAND2_X1 \
  --image docker.io/lizhangmai/iclayout-bench-tools:latest

Results go to results/experiment/<task-id>/<timestamp>/, including the verdict, model usage, retained logs and available GDS/preview assets. Omit --case to run the configured selection, or add --output /path/to/results to select an output root. See the running guide for remote participation, multiple conditions and recovery.

How It Works

flowchart LR
    Task["Circuit + constraints + PDK"] --> Agent["Agent + EDA tools"]
    Agent --> GDS["Frozen GDS"]
    GDS --> Judge["Independent evaluation"]
    Judge --> Result["Score + evidence"]

The Agent performs device implementation, placement, routing and iteration. The evaluator checks the accepted layout snapshot in a separate trusted environment. Scores range from 0 to 100, measuring physical and electrical quality against each task's declared targets. A feasible layout can still have room to improve.

Documentation

FAQ

License

The framework is MIT licensed. Task materials, PDKs and EDA tools retain their own licenses; consult the selected Dataset's notices for material rights.

Repository · Report an Issue · Contribute