EmoNet: A PyTorch port

This package contains a PyTorch port of the EmoNet network originally developed in MatLab described in the paper "Emotion schemas are embedded in the human visual system" by Krager et al., 2019. It is being distributed with explicit permission from the original (first) author.

The original model can be found at: https://github.com/ecco-laboratory/EmoNet
A PyTorch port by this lab is also available at: https://github.com/ecco-laboratory/emonet-pytorch.

Installation

To install this repository, either clone the GitLab project, or install using pip install emonet-py.

Contents

The data folder contains:

• The original model parameters, as exported from MatLab (*.bz2 files).
• The original mean pixel values used to preprocess images (img_mean.txt).
• Two demo images to verify the integrity of the port, i.e., that the outputs generated by the PyTorch model closely match the original MatLab model outputs (demo_*.jpg).
• A PyTorch state_dict object containing the PyTorch translation of the original weights, to be used in conjunction with an AlexNetBig instance to obtain the EmoNet model (emonet.pth).

The package emonet_py contains the following scripts:

• alexnet_big.py: defines the original AlexNet model, compared to the updated version that comes with torchvision.models.
• check_data_files.py: a script that will check whether the data folder, and all expected files in it, are present. If not, they will be downloaded automatically from the GitLab repository.
• convert_emonet_matlab_weights.py: this script can be used to translate the MatLab model parameters to PyTorch. See its internal documentation for details on this process.
• A demonstration script showing how to use the model (demo.py).
• emonet.py: the script defining the EmoNet model, as well as a class, EmoNetPreProcess, to load and preprocess images using the same image normalization used by the original MatLab model.
• emonet_arousal.py: an arousal prediction model, consisting of an extra linear layer following the EmoNet output layer (see paper).
• emonet_valence.py: a valence prediction model, consisting of an extra linear layer following the EmoNet output layer (see paper).
• test_integrity.py: a UnitTest to check the integrity of the ported model. Note that the arousal and valence models are also ports of the original models.

Usage

To load and use EmoNet, simply do (see emonet.py/demo.py):

import os
from emonet_py.emonet import EmoNet, EmoNetPreProcess
from emonet_py.emonet_arousal import EmoNetArousal
from emonet_py.emonet_valence import EmoNetValence

if __name__ == '__main__':
    emonet = EmoNet(b_eval=True)
    emonet_pp = EmoNetPreProcess()
    img_big = os.path.join('..', 'data', 'demo_big.jpg')
    img_loaded = emonet_pp(img_big)
    pred = emonet.emonet(img_loaded.unsqueeze(0))
    emonet.prettyprint(pred, b_pc=True)

    emo_aro = EmoNetArousal()
    print(f"Arousal: {emo_aro(img_loaded.unsqueeze(0))}")

    emo_val = EmoNetValence()
    print(f"Valence: {emo_val(img_loaded.unsqueeze(0))}")

Licensing

This repository is made available under an MIT license (see LICENSE.md). This is in agreement with the original repository, which also uses an MIT license.

Author: Laurent Mertens
Mail: laurent.mertens@kuleuven.be