ColorTransferLib 1.0.0.post4

This library provides color and tyle transfer algorithms which were published in scientific papers. Additionall a set of IQA metrics are available.

ColorTransferLib

The ColorTransferLib is a library focused on color transfer, featuring a range of published algorithms. Some algorithms have been re-implemented, while others are integrated from public repositories. The primary objective of this project is to compile all existing color and style transfer methods into one library with a standardized API. This aids the research community in both development and comparison of algorithms. Currently, the library supports 15 color and style transfer methods for images (PNG-Format), 3D point clouds (PLY-Format), and textured triangle meshes (OBJ-Format with corresponding MTL and PNG). Additionally, it includes 20 metrics for evaluating color transfer results. A detailed list of all algorithms is available below.

API

For seamless integration, adhere to the API specifications of the new color transfer algorithm, depicted in the Figure below.

Each class demands three inputs: Source, Reference, and Options. The Source and Reference should be of the Image or Mesh class type, with the latter encompassing 3D point clouds and textured triangle meshes. The Options input consists of dictionaries, stored as a JSON file in the Options folder. For a sample option, see Listings 1. Every option details an adjustable parameter for the algorithm.

Save each new color transfer class in the ColorTransferLib Repository under the Algorithms folder. This ensures its automatic integration into the user interface. The class should have two essential functions: get_info() and apply(...). The get_info() function yields vital details about the algorithm (refer to Listing 2). It also provides data type details, facilitating the identification of compatible objects for the algorithm. The apply(...) function ingests the inputs and embodies the core logic for color transfer.

The output should resemble a dictionary format, as outlined in Listing 3. A status code of 0 signifies a valid algorithm output, while -1 indicates invalidity. The process time denotes the algorithm's execution duration, useful for subsequent evaluations. The 'object' key in the dictionary holds the result, which should match the class type of the Source input.

Installation

Requirements

(1) Download the Models-Folder from here and place it in the project's root directory.

(2) Install the following packages:

sudo apt-get install liboctave-dev

(3) Install the following octave package:

# activate octave environment
octave
# install packages
octave:1> pkg install -forge image
octave:2> pkg install -forge statistics

(4) Run the gbvs_install.m to make the evaluation metric VSI runnable:

user@pc:~/<Project Path>/ColorTransferLib/Evaluation/VIS/gbvs$ ocatve
octave:1> gbvs_install.m

Install via PyPI

pip install colortransferlib

Install from source

pip install -r requirements/requirements.txt
python setup.py bdist_wheel
pip install --force-reinstall ../ColorTransferLib/dist/ColorTransferLib-0.0.2-py3-none-any.whl 

Usage

from ColorTransferLib.ColorTransfer import ColorTransfer
from ColorTransferLib.ImageProcessing.Image import Image

src = Image(file_path='/media/source.png')
ref = Image(file_path='/media/reference.png') 

algo = "GLO"
ct = ColorTransfer(src, ref, algo)
out = ct.apply()

# No output file extension has to be given
if out["status_code"] == 0:
    out["object"].write("/media/out")
else:
    print("Error: " + out["response"])

Available Color Transfer Methods:

The following color and style transfer methods are integrated in the library. Some of them are reimplemented based on the algorithm's description in the the published papers and others are adopted from existing repositories and adpated to fit the API. The original implementation of the latter methods can be found next to the publication's name. The superscripts (2D, 3D) indicated wether the algorithm is applicable to 2D structures like images and mesh textures or to 3D structures like point clouds. The subscript (CT, ST) describs wether the algorithm is a color or a style transfer algorithm.

Year	ID	Publication
2001	$GLO^{2D,3D}_{CT}$	Color Transfer between Images
2003	$BCC^{2D,3D}_{CT}$	A Framework for Transfer Colors Based on the Basic Color Categories
2005	$PDF^{2D,3D}_{CT}$	N-dimensional probability density function transfer and its application to color transfer
2006	$CCS^{2D,3D}_{CT}$	Color transfer in correlated color space
2007	$MKL^{2D,3D}_{CT}$	The Linear Monge-Kantorovitch Linear Colour Mapping for Example-Based Colour Transfer
2009	$GPC^{2D}_{CT}$	Gradient-Preserving Color Transfer
2010	$FUZ^{2D,3D}_{CT}$	An efficient fuzzy clustering-based color transfer method
2015	$NST^{2D}_{ST}$	A Neural Algorithm of Artistic Style - Original Implementation
2017	$DPT^{2D}_{ST}$	Deep Photo Style Transfer - Original Implementation
2019	$TPS^{2D}_{CT}$	L2 Divergence for robust colour transfer - Original Implementation
2020	$HIS^{2D}_{CT}$	Deep Color Transfer using Histogram Analogy - Original Implementation
2020	$PSN^{3D}_{ST}$	PSNet: A Style Transfer Network for Point Cloud Stylization on Geometry and Color - Original Implementation
2020	$EB3^{3D}_{CT}$	Example-Based Colour Transfer for 3D Point Clouds
2021	$CAM^{2D}_{ST}$	CAMS: Color-Aware Multi-Style Transfer - Original Implementation
2021	$RHG^{2D}_{CT}$	HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms

Available Objective Evaluation Metrics

Three classes of evaluation metrics are considered here. Metrics that evaluate the color consistency with the reference image (indicated with $^r$), metrics that evaluate the structural similarity with the source image (indicated with $^s$) and metrics that evaluates the overall quality of the output (indicated with $^o$).

Year	ID	Name	Publication
/	$PSNR^s_{rgb}$	Peak Signal-to-Noise Ratio	/
/	$HI^r_{rgb}$	Histogram Intersection	/
/	$Corr^r_{rgb}$	Correlation	/
/	$BA^r_{rgb}$	Bhattacharyya Distance	/
/	$MSE^s_{rgb}$	Mean-Squared Error	/
/	$RMSE^s_{rgb}$	Root-Mean-Squared Error	/
2003	$CF^o_{rgyb}$	Colorfulness	Measuring Colourfulness in Natural Images
2003	$MSSSIM^s_{rgb}$	Multi-Scale Structural Similarity Index	Multiscale structural similarity for image quality assessment
2004	$SSIM^s_{rgb}$	Structural Similarity Index	Image quality assessment: from error visibility to structural similarity
2006	$GSSIM^s_{rgb}$	Gradient-based Structural Similarity Index	Gradient-Based Structural Similarity for Image Quality Assessment
2010	$IVSSIM^s_{rgb}$	4-component Structural Similarity Index	Content-partitioned structural similarity index for image quality assessment
2011	$IVEGSSIM^s_{rgb}$	4-component enhanced Gradient-based Structural Similarity Index	An image similarity measure using enhanced human visual system characteristics
2011	$FSIM^s_{c,yiq}$	Feature Similarity Index	FSIM: A Feature Similarity Index for Image Quality Assessment
2012	$BRISQUE^o_{rgb}$	Blind/Referenceless Image Spatial Quality Evaluator	No-Reference Image Quality Assessment in the Spatial Domain
2013	$NIQE^o_{rgb}$	Naturalness Image Quality Evaluator	Making a “Completely Blind” Image Quality Analyzer
2014	$VSI^s_{rgb}$	Visual Saliency-based Index	VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment
2016	$CTQM^{sro}_{lab}$	Color Transfer Quality Metric	Novel multi-color transfer algorithms and quality measure
2018	$LPIPS^s_{rgb}$	Learned Perceptual Image Patch Similarity	The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
2018	$NIMA^o_{rgb}$	Neural Image Assessment	NIMA: Neural Image Assessment
2019	$CSS^{sr}_{rgb}$	Color and Structure Similarity	Selective color transfer with multi-source images

Citation

If you utilize this code in your research, kindly provide a citation:

@inproceeding{potechius2023,
  title={A software test bed for sharing and evaluating color transfer algorithms for images and 3D objects},
  author={Herbert Potechius, Thomas Sikora, Gunasekaran Raja, Sebastian Knorr},
  year={2023},
  booktitle={European Conference on Visual Media Production (CVMP)},
  doi={10.1145/3626495.3626509}
}