ilikepaintings 0.0.2

A package for predicting painting appreciation from images using a linear regressor on top of a frozen CLIP model

I Like Paintings

I Like Paintings is a Python package that can predict the painting appreciation (if people like or not the painting) from the image using a linear regressor trained on top of a frozen CLIP features extractor model on two paintings appreciation datasets. The package contains pre-trained weights for nine models on two datasets (VAPS-999 and Sidhu's one). All linear predictors in this package assume that their input embeddings are L2 normalized.

Installation

You can install I Like Paintings via pip:

pip install ilikepaintings

Usage

To use the package, you need to load the weights of the linear regressor. Here is an example code snippet:

from ilikepaintings import load_predictor, VALID_MODELS, VALID_DATASETS

# Choose a model and dataset
clip_model_name = VALID_MODELS[0]
dataset_name = VALID_DATASETS[0]

# Load the predictor
predictor = load_predictor(clip_model_name, dataset_name)

For a fully working example on a real painting, look at demo.ipynb or try on !

Benchmark

The following table shows the cross-validation scores (k=5) of the linear model with the best hyper-parameters, measured using the Pearson correlation coefficient. These scores were obtained for each CLIP model backend and dataset. The final released models were retrained using the same hyper-parameters and fitted on the whole dataset. All embeddings are L2 normalized.

CLIP	VAPS-999	Sidhu
RN50	0.6785	0.8039 🥉
RN50x4	0.7063	0.7858
RN50x16	0.7307 🥉	0.8150 🥈
RN50x64	0.7335 🥈	0.8199 🥇
RN101	0.6904	0.7964
ViT-B/16	0.6925	0.7863
ViT-B/32	0.6907	0.7823
ViT-L/14	0.7265	0.7941
ViT-L-14@336px	0.7382 🥇	0.7985

License

The license associated with this package is the Creative Commons Attribution 4.0 International License CC-BY 4.0, WITH the following additional conditions:

Before using any part or whole of this package, you must star :star: this repository.

If you use any file associated with this package, including the provided weights or results, you must cite the package using the following BibTeX citation:

@software{ilikepaintings,
  author = {BIZZOZZERO, Nicolas and BENDIDI, Ihab and RISSER-MAROIX, Olivier},
  title = {I Like Paintings},
  url = {https://github.com/VieVie31/i-like-paintings},
  version = {0.0.1},
  date = {2023-03-16},
}

The package authors make no representations or warranties of any kind concerning the code or the information contained herein. They disclaim all liability and responsibility for any errors or omissions in the code, or for any loss, injury, or damage resulting from the use of this package, including but not limited to indirect or consequential damages.