DetFill-with-DHT: Deterministic Hints and Hint-AUC for Line Art Colorization

Official implementation of "Hint-AUC: Deterministic Region-based Hint Generation for Line Art Colorization Evaluation" (Koki Madono, Yuan Mingcheng, Edgar Simo-Serra — IEEE Transactions on Visualization and Computer Graphics, 2026).

Highlights

• Deterministic hints (DHT). The same illustration always yields the same scribble and dot hints, so hint-based colorization methods can be compared without random-seed luck.
• Hint-AUC. One number that scores a model over the whole range of hint ratios, from no hints to fully hinted.
• DetFill. A pixel-space diffusion colorization model trained with these hints, with all checkpoints released.
• Reproducible. Tables are rebuilt from released metric files, every experiment can be re-run on example images with one command, and the paper's key figure is reproduced end-to-end by a single script.

News

• 2026-09 — Code, checkpoints, data and the full reproduction package are public; hintauc 0.1.0 is on PyPI. The paper is accepted at IEEE TVCG.

Installation

pip install hintauc                 # the library: deterministic hints, metrics, Hint-AUC  (PyPI: https://pypi.org/project/hintauc/)
pip install "hintauc[perceptual]"   # + the perceptual metrics (LPIPS / OpenCLIP / DINOv2 / DreamSim)

Latest development version: pip install git+https://github.com/MADONOKOUKI/DetFill-with-DHT.git. For training or inference with DetFill, or for reproducing the paper, use the conda environment replicability/environment.yml (Python 3.9, PyTorch 2.5.1, CUDA 12.4; it also runs on CPU).

Quick start

import hintauc

hints = hintauc.generate_hints("illustration.png", size=64)        # image -> deterministic hints
color, mask = hints.at_ratio(0.10, hint_type="scribble")           # hints of the largest 10 % of the regions
hints.save("out/illustration")                                     # files in the DetFill data layout

evaluator = hintauc.Evaluator(metrics=("psnr", "lpips", "dreamsim"))
print(evaluator("colorized.png", "ground_truth.png"))              # the paper's metrics

Command line: hintauc generate image.png --ratio 0.1 and hintauc eval pred_dir gt_dir --metrics mse psnr ssim. DetFill inference over the hint-ratio grid: cd detfill && GPU=0 bash run_inference.sh scribble (data layout and options in detfill/README.md).

Model zoo

Data: the stored hint maps of the 3,000 test images (v1.0), the test-split line art from the three extractors, the alternative-segmenter hint maps, the example bundle and the user-study stimuli (v1.3). Every file with size and SHA-256: checkpoints/README.md.

Reproducing the paper

All scripts fetch what they need from the releases (SHA-256 verified). Requirements: Linux, Miniconda, ~10 GB of disk, an NVIDIA GPU (CPU works but is slow).

Hardware behind the timings. Machine A: NVIDIA RTX A6000 (48 GB), 2 × AMD EPYC 9124 (32 threads), 377 GB RAM. Machine B: NVIDIA GeForce RTX 2080 Ti (11 GB), 2 × Intel Xeon Gold 6226R (32 threads), 187 GB RAM. Both Ubuntu 22.04, driver 535, CUDA 12.2. Fig. 9: 3 min on A, 4–5 min on B, 30 min on the CPUs of A (16 threads). Example suite and the full-scale estimates: measured on B; an A6000-class GPU is roughly twice as fast.

Results are compared automatically with the paper's numbers, the paper's archived images of the same examples and the authors' reference run. Guide, coverage table and the list of what is bit-exact and what is not: reproduce/README.md. Background on every component: detail_explanation.md.

Added features (since the paper) for improving our library

• hintauc pip library and command line (hint generation, the seven metrics, Hint-AUC).
• Dependency-free, bit-exact longest path path_method="geodesic"; dot options medoid / mean / nearest_mean; region tie-breaking tie_break="stable".
• Protocol switch hint_order: area | label (Table II vs. Table III) in the DetFill loader; training option include_full_hint.
• Metrics beyond the paper's seven, enabled by name: MAE, MS-SSIM, CIEDE2000 colour difference (ΔE00), LPIPS-VGG, DISTS per image, and set-level FID / KID between two directories (hintauc eval pred_dir gt_dir --set_metrics fid kid). Hint-AUC works with any of them.
• DreamSim and the other metric backbones run on CPU-only machines; their weights are cached under ~/.cache/hintauc.
• CPU inference, a flat user-configurable data layout, deterministic region-id colours, other segmenters in the generator.
• The replicability script, the reproduction package and the additional releases (v1.1–v1.3, legacy-2024).

Small documented differences between the paper's text and the released code/data are listed in detail_explanation.md.

Repository structure

replicability/     one-command reproduction of Fig. 9
reproduce/         reproduction package: scripts/, examples/, expected/ (released metrics), data/, paper_experiments/
hintauc/           the library: hints.py, metrics.py, auc.py, cli.py
detfill/           the DetFill model (BBDM fork): training, inference, configs
hint_generation/   original research scripts behind the library
evaluation/        original evaluation scripts
checkpoints/       inventory of all released files with hashes
assets/            images for this page; representative image for the Replicability Stamp

Citation

@article{madono2026hintauc,
  title   = {Hint-AUC: Deterministic Region-based Hint Generation for Line Art Colorization Evaluation},
  author  = {Madono, Koki and Mingcheng, Yuan and Simo-Serra, Edgar},
  journal = {IEEE Transactions on Visualization and Computer Graphics},
  year    = {2026},
  doi     = {10.1109/TVCG.2026.3738401}
}

License and acknowledgements

MIT License; third-party components in THIRD_PARTY_NOTICES.md. detfill/ is built on BBDM (MIT). Released data files are derived from Danbooru2021 illustrations and are provided for non-commercial research use only.