libreface 0.2.0

Commandline

You can use this package through the command line using the following command.

libreface --input_path="path/to/your_image_or_video"

Note that the above script would save results in a CSV at the default location - sample_results.csv. If you want to specify your own path, use the --output_path command line argument,

libreface --input_path="path/to/your_image_or_video" --output_path="path/to/save_results.csv"

To change the device used for inference, use the --device command line argument,

libreface --input_path="path/to/your_image_or_video" --device="cuda:0"

To save intermediate files, libreface uses a temporary directory that defaults to ./tmp. To change the temporary directory path,

libreface --input_path="path/to/your_image_or_video" --temp="your/temp/path"

For video inference, our code processes the frames of your video in batches. You can specify the batch size and the number of workers for data loading as follows,

libreface --input_path="path/to/your_video" --batch_size=256 --num_workers=2 --device="cuda:0"

Note that by default, the --batch_size argument is 256, and --num_workers argument is 2. You can increase or decrease these values according to your machine’s capacity.

Examples

Download a sample image from our GitHub repository. To get the facial attributes for this image and save to a CSV file, simply run,

libreface --input_path="sample_disfa.png"

Download a sample video from our GitHub repository. To run the inference on this video using a GPU and save the results to my_custom_file.csv run the following command,

libreface --input_path="sample_disfa.avi" --output_path="my_custom_file.csv" --device="cuda:0"

Note that for videos, each row in the saved CSV file corresponds to individual frames in the given video.

Python API

Here’s how to use this package in your Python scripts.

To assign the results to a Python variable,

import libreface
detected_attributes = libreface.get_facial_attributes(image_or_video_path)

To save the results to a csv file,

import libreface
libreface.get_facial_attributes(image_or_video_path,
                                output_save_path = "your_save_path.csv")

To change the device used for inference, use the device parameter,

import libreface
libreface.get_facial_attributes(image_or_video_path,
                                device = "cuda:0") # can be "cpu" or "cuda:0", "cuda:1", ...

To save intermediate files, libreface uses a temporary directory that defaults to ./tmp. To change the temporary directory path,

import libreface
libreface.get_facial_attributes(image_or_video_path,
                                temp_dir = "your/temp/path")

For video inference, our code processes the frames of your video in batches. You can specify the batch size and the number of workers for data loading as follows,

import libreface
libreface.get_facial_attributes(video_path,
                                batch_size = 256,
                                num_workers = 2)

Note that by default, the batch_size is 256, and num_workers is 2. You can increase or decrease these values according to your machine’s capacity.

Gaze Estimation

Gaze yaw and pitch are returned by default in the get_facial_attributes output (keys gaze_yaw and gaze_pitch, or columns of the same name for video).

The figure above shows a Gaze360 example with three panels: biased (raw) model output (red, yaw=+11.07°, pitch=+11.43°), unbiased (bias-corrected) prediction (green, yaw=+4.51°, pitch=+6.44°), and ground truth (blue, yaw=-7.68°, pitch=+2.61°). Bias correction subtracts the systematic offset measured on the held-out test set (+6.55° yaw, +5.05° pitch).

If you only need gaze and want to skip AU and expression inference, call estimate_gaze (image) or estimate_gaze_video (list of aligned frames) directly. Both expect an already aligned face image — use libreface.get_aligned_image to produce one from a raw image.

import libreface

aligned_image_path, _, _ = libreface.get_aligned_image("path/to/your_image.png", temp_dir="./tmp")
gaze = libreface.estimate_gaze(aligned_image_path, device="cpu")
print(gaze)  # {'gaze_yaw': <float>, 'gaze_pitch': <float>}

For videos, pass a list of aligned frame paths:

import libreface

gaze_df = libreface.estimate_gaze_video(aligned_frames_path_list,
                                        device="cuda:0",
                                        batch_size=256,
                                        num_workers=2)
# gaze_df has columns "gaze_yaw" and "gaze_pitch", one row per frame.