Skip to main content

Facial expression recognition from images

Project description

FER

Facial expression recognition.

image

PyPI version Build Status Downloads

INSTALLATION

Currently FER only supports Python 3.6 onwards. It can be installed through pip:

$ pip install fer

This implementation requires OpenCV>=3.2 and Tensorflow>=1.7.0 installed in the system, with bindings for Python3.

They can be installed through pip (if pip version >= 9.0.1):

$ pip install tensorflow>=1.7 opencv-contrib-python==3.3.0.9

or compiled directly from sources (OpenCV3, Tensorflow).

Note that a tensorflow-gpu version can be used instead if a GPU device is available on the system, which will speedup the results. It can be installed with pip:

$ pip install tensorflow-gpu\>=1.7.0

USAGE

The following example illustrates the ease of use of this package:

from fer import FER
import cv2

img = cv2.imread("justin.jpg")
detector = FER()
detector.detect_emotions(img)

Sample output:

OrderedDict([{'box': [277, 90, 48, 63], 'emotions': {'angry': 0.02, 'disgust': 0.0, 'fear': 0.05, 'happy': 0.16, 'neutral': 0.09, 'sad': 0.27, 'surprise': 0.41}])

Just want the top emotion? Try:

emotion, score = detector.top_emotion(img) # 'happy', 0.99

MTCNN Facial Recognition

Faces by default are detected using OpenCV's Haar Cascade classifier. To use the more accurate MTCNN network, add the parameter:

detector = FER(mtcnn=True)

Video

For recognizing facial expressions in video, the Video class splits video into frames. It can use a local Keras model (default) or Peltarion API for the backend:

from fer import Video
from fer import FER

video_filename = "tests/woman2.mp4"
video = Video(video_filename)

# Analyze video, displaying the output
detector = FER(mtcnn=True)
raw_data = video.analyze(detector, display=True)
df = video.to_pandas(raw_data)

The detector returns a list of JSON objects. Each JSON object contains two keys: 'box' and 'emotions':

  • The bounding box is formatted as [x, y, width, height] under the key 'box'.
  • The emotions are formatted into a JSON object with the keys 'anger', 'disgust', 'fear', 'happy', 'sad', surprise', and 'neutral'.

Other good examples of usage can be found in the files example.py and video-example.py located in the root of this repository.

PELTARION API

Add the API URL as an emotion_model argument to FER():

detector = FER(emotion_model=MY_API_URL)

FER will then pick up the URL and token from the environment as EMOTION_API_URL and EMOTION_API_TOKEN.

MODEL

FER bundles a Keras model, as well as support for Peltarion API.

The model is a convolutional neural network with weights saved to HDF5 file in the data folder relative to the module's path. It can be overriden by injecting it into the FER() constructor during instantiation with the emotion_model parameter.

LICENSE

MIT License.

CREDIT

This code includes methods and package structure copied or derived from Iván de Paz Centeno's implementation of MTCNN and Octavia Arriaga's facial expression recognition repo.

REFERENCE

FER 2013 dataset curated by Pierre Luc Carrier and Aaron Courville, described in:

"Challenges in Representation Learning: A report on three machine learning contests," by Ian J. Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, Yingbo Zhou, Chetan Ramaiah, Fangxiang Feng, Ruifan Li, Xiaojie Wang, Dimitris Athanasakis, John Shawe-Taylor, Maxim Milakov, John Park, Radu Ionescu, Marius Popescu, Cristian Grozea, James Bergstra, Jingjing Xie, Lukasz Romaszko, Bing Xu, Zhang Chuang, and Yoshua Bengio, arXiv:1307.0414.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for fer, version 19.0.6
Filename, size File type Python version Upload date Hashes
Filename, size fer-19.0.6-py3-none-any.whl (810.4 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size fer-19.0.6.tar.gz (808.0 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page