SeetaPsych Emo

Facial affect/emotion recognition modules for SeetaPsych

简体中文 | English

Usage

This project is already included in the seetapsych-lib default configuration. Download and use it via seetapsych-manager download.

For usage, refer to SeetaPsych.

You can additionally add this algorithm module using the following methods.

WebUI

Run seetapsych-webui with the --dirs argument to use it.

seetapsych-webui --files seetapsych_emo/modules/emonet.yml

Programmatic Usage

Add the following code in your program to use this algorithm module.

from seetapsych_lib.runtime.factory import Factory
from seetapsych_lib.runtime.pipeline import Pipeline

factory = Factory()
factory.load_file_modules("seetapsych_emo/modules/emonet.yml")

pipeline = Pipeline(factory, ...)

pipeline.add_attributes("face/expression", "face/action_units", "face/dimensional_affect")

For a complete end-to-end example with visualization, see examples/image_emonet.py.

Module Catalog

SeetaEmoNet

Multi-task facial affect estimation: action units, categorical expressions, and continuous valence-arousal dimensions.

Figure 1. SeetaEmoNet output visualization — predicted AUs, expressions, and valence-arousal.

Module config: emonet.yml

Description: Unified multi-task MAE-ViT model predicting 16 AUs, 7 expressions, and valence-arousal simultaneously from 5-point aligned face crops.

Parameters: (none)

Models

Output Attributes

• face/action_units — spec.
• face/expression — spec.
• face/dimensional_affect — spec.

Output details:

• face/expression: 7 basic expression classes with confidence scores — neutral, anger, disgust, fear, happy, sad, surprise.
• face/action_units: 16 Facial Action Units with intensities — AU1, AU2, AU4, AU5, AU6, AU7, AU9, AU10, AU12, AU15, AU17, AU20, AU23, AU24, AU25, AU26.
• face/dimensional_affect: Continuous valence and arousal values in a dict {valence: float, arousal: float}.