widget2code-bench 0.1.1

Benchmark evaluation for widget code generation — 12 quality metrics across layout, legibility, perceptual, style, and geometry.

widget2code-bench

Benchmark evaluation for widget code generation — 12 quality metrics across layout, legibility, perceptual, style, and geometry.

Installation

# 1. Install PyTorch with CUDA support first (skip if CPU-only)
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126

# 2. Install widget2code-bench
pip install widget2code-bench

Note: PyPI only ships CPU-only PyTorch. To use --cuda, you must install PyTorch from the official index before installing this package.

Usage

Single image mode

Evaluate one GT-prediction pair. Prints JSON results to stdout, no files saved.

widget2code-bench \
  --gt_image /path/to/gt.png \
  --pred_image /path/to/pred.png \
  --cuda

Batch mode

Evaluate all matched pairs in directories.

widget2code-bench \
  --gt_dir /path/to/GT \
  --pred_dir /path/to/predictions \
  --pred_name output.png \
  --cuda

Directory Structure (batch mode)

• GT dir: flat image files with 4-digit IDs in filenames (e.g. gt_0001.png)
• Pred dir: subfolders with 4-digit IDs in names, each containing --pred_name file

gt_dir/                     pred_dir/
  gt_0001.png                 image_0001/
  gt_0002.png                   output.png
  ...                         image_0002/
                                output.png

Options

Flag	Default	Description
--gt_image	—	Single GT image path
--pred_image	—	Single prediction image path
--gt_dir	—	GT directory (flat image files)
--pred_dir	—	Prediction directory (subfolders)
--pred_name	output.png	Prediction filename inside each subfolder
--output_dir	{pred_dir}/.analysis	Statistics output directory
--workers	4	Parallel threads
--cuda	off	Enable GPU
--skip_eval	off	Skip evaluation, only generate statistics

Output (batch mode)

1. Evaluation — Saves evaluation.json in each prediction subfolder + evaluation.xlsx in pred_dir
2. Statistics — Saves metrics_stats.json and metrics.xlsx to {pred_dir}/.analysis/

License

Apache-2.0