SeetaPsych Gaze Screen

Screen gaze estimation modules for SeetaPsych

Usage

This project is already included in the seetapsych-lib default configuration. Download and use it via seetapsych-manager download.

For usage, refer to SeetaPsych.

The gaze estimation algorithms depend on open-gaze-estimation. Please install it from GitHub separately:

uv pip install git+https://github.com/Elorfiniel/open-gaze-estimation-2025-release.git

You can additionally add this algorithm module using the following methods.

WebUI

Run seetapsych-webui with the --files argument to use it.

seetapsych-webui --files \
  seetapsych_gaze_screen/modules/affnet.yml \
  seetapsych_gaze_screen/modules/tdgazenet.yml

Programmatic Usage

Add the following code in your program to use this algorithm module.

from seetapsych_lib.runtime.factory import Factory
from seetapsych_lib.runtime.pipeline import Pipeline

factory = Factory()
factory.load_file_modules("seetapsych_gaze_screen/modules/affnet.yml")

pipeline = Pipeline(factory, ...)

pipeline.add_attributes("face/gaze_screen")

Module Catalog

GazeScreen-AFFNet(OpenGaze)

Open-source gaze estimation toolkit providing screen gaze coordinates from facial landmarks or mesh.

Module config: affnet.yml

Description

Estimate screen gaze coordinates using the AFFNet¹, which predicts a single gaze location shared by both eyes. Suitable for single-user desktop scenarios with moderate accuracy and computational cost.

Parameters

Models

GazeScreen-TDGazeNet(OpenGaze)

Open-source gaze estimation toolkit providing screen gaze coordinates from facial landmarks or mesh.

Module config: tdgazenet.yml

Description

High-accuracy gaze estimation via TdGazeNet with 3D face prior, camera intrinsics and multi-task head; outputs distinct per-eye gaze vectors for left and right eye. Highest accuracy among OpenGaze variants at the cost of heavier compute.

Parameters

Models

data parameter reference

All gaze-screen modules share the same top-level structure under the data parameter. Two layouts are used depending on whether the algorithm relies on a 3D face prior. Screen dimensions (w_px, h_px, w_mm, h_mm) are common to both layouts; camera-section fields differ.

Common fields (screen section)

Screen physical dimensions and pixel resolution are used to convert between camera-space millimeters and screen-space pixels. Values must match the actual monitor used for the experiment.

Layout A — Simple camera offset (AFFNet)

Used by algorithms that estimate gaze in the camera coordinate frame directly, then project it onto the screen plane from a known relative position. No per-pixel lens distortion is applied; if your camera has strong distortion, undistort frames before feeding them into the pipeline.

{
  "camera": {
    "screen_x_mm": -155,
    "screen_y_mm": -5
  },
  "screen": {
    "h_px": 1080,
    "w_px": 1920,
    "h_mm": 174,
    "w_mm": 310
  }
}

Layout B — Full camera calibration (TDGazeNet)

Used by algorithms that leverage a 3D face prior. They require explicit camera intrinsics, an extrinsic screen-to-camera transform, and OpenCV-format distortion coefficients to lift 2D landmarks back into metric 3D space before projecting the gaze ray onto the screen plane. Run a standard OpenCV/Matlab checkerboard calibration once on your capture setup and paste the matrices here.

{
  "camera": {
    "intrinsic": [
      [972.01, 0.0, 652.68],
      [0.0, 972.35, 373.91],
      [0.0, 0.0, 1.0]
    ],
    "extrinsic": [
      [-1.0, 0.0, 0.0, 155.0],
      [0.0, 1.0, 0.0, 5.0],
      [0.0, 0.0, -1.0, 2.5]
    ],
    "distortion": [0.123508, -0.334222, -0.002206, 0.000207, 0.199979]
  },
  "screen": {
    "h_px": 1080,
    "w_px": 1920,
    "h_mm": 174,
    "w_mm": 310
  }
}

data.camera.extrinsic

The camera extrinsic matrix. For the current implementation, only the translation vector $t = [t_x, t_y, t_z]^T$, i.e. the last column of the matrix, needs to be adjusted for the actual camera–screen setup. Leave the other entries unchanged.

The translation vector $t$ gives the position of the top-left corner of the active display area in the camera coordinate system, in millimeters.

The camera coordinate system is centered at the camera optical center:

• +X points to the right in the camera image (to the user's left when facing the monitor).
• +Y points downward.
• +Z points forward along the camera optical axis, from the camera toward the user.

Measure $t_x$, $t_y$, and $t_z$ as the signed offsets from the camera optical center to the top-left corner of the active display area along these axes.

Example. If the camera optical center is horizontally aligned with the center of a $310,\mathrm{mm}$-wide active display area, the top-left corner is approximately $155,\mathrm{mm}$ along +X, so $t_x \approx 155,\mathrm{mm}$. If the top edge of the active display area is $5,\mathrm{mm}$ below the camera optical center, then $t_y \approx 5,\mathrm{mm}$. If the display plane is $2.5,\mathrm{mm}$ along +Z from the camera optical center, then $t_z \approx 2.5,\mathrm{mm}$.

Thus,

$$ t \approx [155,; 5,; 2.5]^T\ \mathrm{mm}. $$

References