SeetaPsych Gaze

Gaze estimation modules for SeetaPsych

Usage

This project is already included in the seetapsych-lib default configuration. Download and use it via seetapsych-manager download.

For usage, refer to SeetaPsych.

The gaze estimation algorithms depend on open-gaze-estimation. Please install it from GitHub separately:

uv pip install git+https://github.com/Elorfiniel/open-gaze-estimation-2025-release.git

You can additionally add this algorithm module using the following methods.

WebUI

Run seetapsych-webui with the --files argument to use it.

seetapsych-webui --files \
  seetapsych_gaze_screen/modules/affnet.yml \
  seetapsych_gaze_screen/modules/tdgazenet.yml

Programmatic Usage

Add the following code in your program to use this algorithm module.

from seetapsych_lib.runtime.factory import Factory
from seetapsych_lib.runtime.pipeline import Pipeline

factory = Factory()
factory.load_file_modules("seetapsych_gaze_screen/modules/affnet.yml")

pipeline = Pipeline(factory, ...)

pipeline.add_attributes("face/gaze_screen")

Module Catalog

GazeScreen-AFFNet(OpenGaze)

Open-source gaze estimation toolkit providing screen gaze coordinates from facial landmarks or mesh.

Module config: affnet.yml

Description

Estimate screen gaze coordinates using the AFFNet¹, which predicts a single gaze location shared by both eyes. Suitable for single-user desktop scenarios with moderate accuracy and computational cost.

Parameters

Models

GazeScreen-TDGazeNet(OpenGaze)

Open-source gaze estimation toolkit providing screen gaze coordinates from facial landmarks or mesh.

Module config: tdgazenet.yml

Description

High-accuracy gaze estimation via TdGazeNet with 3D face prior, camera intrinsics and multi-task head; outputs distinct per-eye gaze vectors for left and right eye. Highest accuracy among OpenGaze variants at the cost of heavier compute.

Parameters

Models

data parameter reference

All gaze-screen modules share the same top-level structure under the data parameter. Two layouts are used depending on whether the algorithm relies on a 3D face prior. Screen dimensions (w_px, h_px, w_mm, h_mm) are common to both layouts; camera-section fields differ.

Common fields (screen section)

Screen physical dimensions and pixel resolution are used to convert between camera-space millimeters and screen-space pixels. Values must match the actual monitor used for the experiment.

Layout A — Simple camera offset (AFFNet)

Used by algorithms that estimate gaze in the camera coordinate frame directly, then project it onto the screen plane from a known relative position. No per-pixel lens distortion is applied; if your camera has strong distortion, undistort frames before feeding them into the pipeline.

{
  "camera": {
    "screen_x_mm": -155,
    "screen_y_mm": -5
  },
  "screen": {
    "h_px": 1080,
    "w_px": 1920,
    "h_mm": 174,
    "w_mm": 310
  }
}

Layout B — Full camera calibration (TDGazeNet)

Used by algorithms that leverage a 3D face prior. They require explicit camera intrinsics, an extrinsic screen-to-camera transform, and OpenCV-format distortion coefficients to lift 2D landmarks back into metric 3D space before projecting the gaze ray onto the screen plane. Run a standard OpenCV/Matlab checkerboard calibration once on your capture setup and paste the matrices here.

{
  "camera": {
    "intrinsic": [
      [972.01, 0.0, 652.68],
      [0.0, 972.35, 373.91],
      [0.0, 0.0, 1.0]
    ],
    "extrinsic": [
      [-1.0, 0.0, 0.0, 155.0],
      [0.0, 1.0, 0.0, 5.0],
      [0.0, 0.0, -1.0, 2.5]
    ],
    "distortion": [0.123508, -0.334222, -0.002206, 0.000207, 0.199979]
  },
  "screen": {
    "h_px": 1080,
    "w_px": 1920,
    "h_mm": 174,
    "w_mm": 310
  }
}

References