Metadata-Version: 2.4
Name: hintauc
Version: 0.2.0
Summary: Deterministic region-based hint generation (DHT) and Hint-AUC evaluation for line-art colorization
Author: Koki Madono
License-Expression: MIT
Project-URL: Homepage, https://github.com/MADONOKOUKI/DetFill-with-DHT
Project-URL: Repository, https://github.com/MADONOKOUKI/DetFill-with-DHT
Project-URL: Issues, https://github.com/MADONOKOUKI/DetFill-with-DHT/issues
Project-URL: Paper, https://doi.org/10.1109/TVCG.2026.3738401
Keywords: line art,colorization,evaluation,hint generation,Hint-AUC,diffusion
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Image Processing
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: <3.13,>=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: opencv-python
Requires-Dist: scikit-image>=0.19
Requires-Dist: fil_finder<1.8,>=1.7
Requires-Dist: astropy<6,>=5.0
Requires-Dist: pillow
Provides-Extra: perceptual
Requires-Dist: torch; extra == "perceptual"
Requires-Dist: torchvision; extra == "perceptual"
Requires-Dist: lpips; extra == "perceptual"
Requires-Dist: open_clip_torch; extra == "perceptual"
Requires-Dist: transformers; extra == "perceptual"
Requires-Dist: dreamsim; extra == "perceptual"
Requires-Dist: torchmetrics; extra == "perceptual"
Requires-Dist: dists-pytorch; extra == "perceptual"
Requires-Dist: torch-fidelity; extra == "perceptual"
Dynamic: license-file

# DetFill-with-DHT: Deterministic Hints and Hint-AUC for Line Art Colorization

[![IEEE TVCG 2026](https://img.shields.io/badge/IEEE%20TVCG-2026-blue)](https://doi.org/10.1109/TVCG.2026.3738401)
[![DOI](https://img.shields.io/badge/DOI-10.1109%2FTVCG.2026.3738401-blue)](https://doi.org/10.1109/TVCG.2026.3738401)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue)](#installation)
[![PyTorch 2.5](https://img.shields.io/badge/PyTorch-2.5-ee4c2c)](#installation)
[![PyPI](https://img.shields.io/pypi/v/hintauc)](https://pypi.org/project/hintauc/)
[![Releases](https://img.shields.io/github/v/release/MADONOKOUKI/DetFill-with-DHT)](https://github.com/MADONOKOUKI/DetFill-with-DHT/releases)

Official implementation of **"Hint-AUC: Deterministic Region-based Hint Generation for Line Art Colorization Evaluation"**
(Koki Madono, Yuan Mingcheng, Edgar Simo-Serra — IEEE Transactions on Visualization and Computer Graphics, 2026).

![Deterministic hint generation pipeline](assets/readme/dht_pipeline.png)

## Highlights

- **Deterministic hints (DHT).** The same illustration always yields the same scribble and dot hints, so
  hint-based colorization methods can be compared without random-seed luck.
- **Hint-AUC.** One number that scores a model over the *whole* range of hint ratios, from no hints to fully hinted.
- **DetFill.** A pixel-space diffusion colorization model trained with these hints, with all checkpoints released.
- **Reproducible.** Tables are rebuilt from released metric files, every experiment can be re-run on example images
  with one command, and the paper's key figure is reproduced end-to-end by a single script.

## News

- **2026-09** — Code, checkpoints, data and the full reproduction package are public; `hintauc` 0.1.0 is on PyPI. The paper is accepted at IEEE TVCG.

## Installation

```bash
pip install hintauc                 # the library: deterministic hints, metrics, Hint-AUC  (PyPI: https://pypi.org/project/hintauc/)
pip install "hintauc[perceptual]"   # + the perceptual metrics (LPIPS / OpenCLIP / DINOv2 / DreamSim)
```

Latest development version: `pip install git+https://github.com/MADONOKOUKI/DetFill-with-DHT.git`.
For training or inference with DetFill, or for reproducing the paper, use the conda environment
`replicability/environment.yml` (Python 3.9, PyTorch 2.5.1, CUDA 12.4; it also runs on CPU).

## Quick start

```python
import hintauc

hints = hintauc.generate_hints("illustration.png", size=64)        # image -> deterministic hints
color, mask = hints.at_ratio(0.10, hint_type="scribble")           # hints of the largest 10 % of the regions
hints.save("out/illustration")                                     # files in the DetFill data layout

evaluator = hintauc.Evaluator(metrics=("psnr", "lpips", "dreamsim"))
print(evaluator("colorized.png", "ground_truth.png"))              # the paper's metrics
```

Command line: `hintauc generate image.png --ratio 0.1` and `hintauc eval pred_dir gt_dir --metrics mse psnr ssim`.
DetFill inference over the hint-ratio grid: `cd detfill && GPU=0 bash run_inference.sh scribble`
(data layout and options in [detfill/README.md](detfill/README.md)).

## Model zoo

| Model | Hints | Base channels | Trained on | Download |
|---|---|---|---|---|
| DetFill (paper) | scribble | 96 | Danbooru2021 | [v1.0](https://github.com/MADONOKOUKI/DetFill-with-DHT/releases/tag/v1.0) `detfill_scribble_illust_200ep.pth` (1.08 GB) |
| DetFill (paper) | dot | 64 | Danbooru2021 | [v1.0](https://github.com/MADONOKOUKI/DetFill-with-DHT/releases/tag/v1.0) `detfill_dot_illust_200ep.pth` (0.48 GB) |
| DetFill, DanbooRegion hints | scribble | 96 | Danbooru2021 | [v1.1](https://github.com/MADONOKOUKI/DetFill-with-DHT/releases/tag/v1.1) |
| DetFill, SLIC hints | scribble | 96 | Danbooru2021 | [v1.1](https://github.com/MADONOKOUKI/DetFill-with-DHT/releases/tag/v1.1) |
| DetFill, natural images | scribble / dot | 96 | ImageNet subset | [v1.2](https://github.com/MADONOKOUKI/DetFill-with-DHT/releases/tag/v1.2) |
| Diffusart retrained with our hints | scribble / dot | — | Danbooru2021 | [v1.2](https://github.com/MADONOKOUKI/DetFill-with-DHT/releases/tag/v1.2) |
| DetFill, channel ablation | scribble | 32 / 64 | Danbooru2021 | [legacy-2024](https://github.com/MADONOKOUKI/DetFill-with-DHT/releases/tag/legacy-2024) |
| DetFill, 2024 submission | scribble / dot | 64 | Danbooru2021 / ImageNet | [legacy-2024](https://github.com/MADONOKOUKI/DetFill-with-DHT/releases/tag/legacy-2024) |

Data: the stored hint maps of the 3,000 test images (v1.0), the test-split line art from the three extractors, the
alternative-segmenter hint maps, the example bundle and the user-study stimuli (v1.3). Every file with size and
SHA-256: [checkpoints/README.md](checkpoints/README.md).

## Reproducing the paper

All scripts fetch what they need from the releases (SHA-256 verified). Requirements: Linux, Miniconda, ~10 GB of disk,
an NVIDIA GPU (CPU works but is slow).

| Goal | Command (no arguments) | Time |
|---|---|---|
| **Fig. 9** end to end (the Replicability-Stamp script) | `bash replicability/run.sh` | GPU ≈ 3 min, CPU ≈ 30 min |
| **Every table** rebuilt from the released metric files | `python reproduce/scripts/A1_tables_from_released_metrics.py` (322/324 cells match)<br>`python reproduce/scripts/A2_userstudy_glmm.py` (all values match) | < 2 min, CPU |
| **Every experiment** re-run on 12 example illustrations | `bash reproduce/examples/run_examples.sh` | smoke ≈ 5 min, default ≈ 1 h, full = hours |
| **A whole table row** on the 3,000 test images | `DATA_ROOT=… bash reproduce/scripts/run_hauc_pipeline.sh` | ≈ 14 GPU-h per row |
| **Training** from scratch | `bash reproduce/scripts/B9_train_detfill.sh` | days, 10 GPUs |

**Hardware behind the timings.** Machine A: NVIDIA RTX A6000 (48 GB), 2 × AMD EPYC 9124 (32 threads), 377 GB RAM.
Machine B: NVIDIA GeForce RTX 2080 Ti (11 GB), 2 × Intel Xeon Gold 6226R (32 threads), 187 GB RAM. Both Ubuntu 22.04,
driver 535, CUDA 12.2. Fig. 9: 3 min on A, 4–5 min on B, 30 min on the CPUs of A (16 threads). Example suite and the
full-scale estimates: measured on B; an A6000-class GPU is roughly twice as fast.

Results are compared automatically with the paper's numbers, the paper's archived images of the same examples and the
authors' reference run. Guide, coverage table and the list of what is bit-exact and what is not:
[reproduce/README.md](reproduce/README.md). Background on every component: [detail_explanation.md](detail_explanation.md).

## Added features (since the paper) for improving our library

- `hintauc` pip library and command line (hint generation, the seven metrics, Hint-AUC).
- Dependency-free, bit-exact longest path `path_method="geodesic"`; dot options `medoid` / `mean` / `nearest_mean`;
  region tie-breaking `tie_break="stable"`.
- Protocol switch `hint_order: area | label` (Table II vs. Table III) in the DetFill loader; training option `include_full_hint`.
- Metrics beyond the paper's seven, enabled by name: MAE, MS-SSIM, CIEDE2000 colour difference (ΔE00), LPIPS-VGG,
  DISTS per image, and set-level FID / KID between two directories (`hintauc eval pred_dir gt_dir --set_metrics fid kid`).
  Hint-AUC works with any of them.
- DreamSim and the other metric backbones run on CPU-only machines; their weights are cached under `~/.cache/hintauc`.
- CPU inference, a flat user-configurable data layout, deterministic region-id colours, other segmenters in the generator.
- The replicability script, the reproduction package and the additional releases (v1.1–v1.3, legacy-2024).

Small documented differences between the paper's text and the released code/data are listed in
[detail_explanation.md](detail_explanation.md#changes-relative-to-the-paper).

## Repository structure

```
replicability/     one-command reproduction of Fig. 9
reproduce/         reproduction package: scripts/, examples/, expected/ (released metrics), data/, paper_experiments/
hintauc/           the library: hints.py, metrics.py, auc.py, cli.py
detfill/           the DetFill model (BBDM fork): training, inference, configs
hint_generation/   original research scripts behind the library
evaluation/        original evaluation scripts
checkpoints/       inventory of all released files with hashes
assets/            images for this page; representative image for the Replicability Stamp
```

## Citation

```bibtex
@article{madono2026hintauc,
  title   = {Hint-AUC: Deterministic Region-based Hint Generation for Line Art Colorization Evaluation},
  author  = {Madono, Koki and Mingcheng, Yuan and Simo-Serra, Edgar},
  journal = {IEEE Transactions on Visualization and Computer Graphics},
  year    = {2026},
  doi     = {10.1109/TVCG.2026.3738401}
}
```

## License and acknowledgements

MIT License; third-party components in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). `detfill/` is built on
[BBDM](https://github.com/xuekt98/BBDM) (MIT). Released data files are derived from Danbooru2021 illustrations and are
provided for non-commercial research use only.
