Skip to main content

SeetaPsych Attributes

Face and body based psychology analysis

SeetaPsych Lib is a Python library for face- and body-based psychology analysis. It provides a modular Pipeline/Runner runtime and an optional Streamlit WebUI.

This project is used to manage the specifications for various attribute outputs, providing a unified standard so that different algorithm implementations can produce interchangeable and reusable module outputs.

TypedDict Type Hints

Alongside the JSON schemas documented below, this project ships a set of ready-to-use TypedDict declarations under seetapsych_attributes.types so that your IDE can provide auto-completions and static type checks directly on the runner's report dict:

# -*- coding: utf-8 -*-
import json

import cv2

from seetapsych_lib.runtime.factory import Factory
from seetapsych_lib.runtime.pipeline import Pipeline
from seetapsych_lib.runtime.runner import Runner

from seetapsych_attributes.types import Report, BBox, FaceDetection


def main():
    factory = Factory()
    factory.load_builtin_modules()

    pipeline = Pipeline(factory, attributes=["face/detection"])
    pipeline.solve()
    pipeline.install_requirements()
    pipeline.cache_models()

    runner = Runner(pipeline)

    report: Report = runner.run(data={"default": cv2.imread("data/a.jpg")})

    # IDE autocompletion + type inference for every attribute key:
    detections: FaceDetection | None = report.get("face_detection")
    if detections:
        first: BBox = detections[0]
        x1, y1, x2, y2 = first["xyxy"]
        score: float = first["score"]
        print(f"face at ({x1},{y1})-({x2},{y2}), score = {score:.3f}")

    print(json.dumps(report, indent=2, ensure_ascii=False))


if __name__ == "__main__":
    main()

The top-level Report TypedDict includes every attribute key defined in the catalog (all fields are optional, because a pipeline may only request a subset). Per-attribute element types such as BBox, Landmarks, Selection, ActionUnits, Expression, HeartRate, HeadSocialGaze, etc. are also exported individually.

Catalog

  • face/detection Face detection results as rectangular bounding boxes.
  • face/landmarks Facial landmarks for basic alignment: L-eye, R-eye, nose, L-mouth, R-mouth (10 interleaved floats).
  • face/selection Selected face PID. Selected face order is reflected in face/detection and face/landmarks.
  • face/action_units Indicate the confidence level of each Action Unit. Not all Action Units' results may be output.
  • face/expression Indicate the confidence level of each expression.
  • face/dense_landmarks 280-point dense facial landmarks (560 interleaved [x,y] floats).
  • face/feature Per-face L2-normalized feature embeddings for recognition, clustering, or similarity search. Each inner list corresponds to one aligned face in face/landmarks order; vector dimension is algorithm-specific (typically 512 for ArcFace).
  • face/mesh 468-point 3D face mesh landmarks in normalized coordinates.
  • face/gaze_screen Per-eye screen-space point-of-gaze coordinates in pixels and camera-space point-of-gaze coordinates in millimeters. Camera-space coordinates may be 2D or 3D depending on the algorithm; origin is at the camera center.
  • face/heart_rate Heart rate (BPM) estimated from buffered face video frames.
  • face/dimensional_affect Continuous valence-arousal affect dimensions alongside discrete expressions and Action Units.
  • head/detection Multi-person head bounding box detection results.
  • head/selection Top-N head selection result (count + original indices), reordering head_detection.
  • head/gaze_point Per-head 2D scene gaze target point with associated likelihood heatmap.
  • head/social_gaze Dyadic social gaze relations between two detected people. Class set: share, mutual, single, miss, void.

face/detection

Face detection results as rectangular bounding boxes.

Properties

  • face_detection (array, required)
    • Items: Refer to BBox.

Definitions

  • BBox (object)
    • xyxy (array, required): Length must be equal to 4.
      • Items (number)
    • score (number, required)

Examples

{
    "face_detection": [
        {
            "score": 0.5,
            "xyxy": [
                100,
                200,
                300,
                400
            ]
        }
    ]
}

face/landmarks

Facial landmarks for basic alignment: L-eye, R-eye, nose, L-mouth, R-mouth (10 interleaved floats).

Properties

  • face_landmarks (array, required)

Definitions

  • Landmarks (object)
    • landmarks (array, required): Length must be equal to 10.
      • Items (number)

Examples

{
    "face_landmarks": [
        {
            "landmarks": [
                100,
                100,
                200,
                200,
                300,
                300,
                400,
                400,
                500,
                500
            ]
        }
    ]
}

face/selection

Selected face PID. Selected face order is reflected in face/detection and face/landmarks.

Properties

  • face_selection (required): Refer to Selection.

Definitions

  • Selection (object)
    • pid (integer, required): PID of selected face detection (1-based).

Examples

{
    "face_detection": [
        {
            "score": 0.5,
            "xyxy": [
                100,
                200,
                300,
                400
            ]
        }
    ],
    "face_selection": {
        "pid": 1
    }
}

face/action_units

Indicate the confidence level of each Action Unit. Not all Action Units' results may be output.

Properties

  • face_action_units (array, required)

Definitions

  • ActionUnits (object)
    • AU1: [0, 1]. Inner Brow Raiser. Default: null.
      • Any of
        • number
        • null
    • AU2: [0, 1]. Outer Brow Raiser. Default: null.
      • Any of
        • number
        • null
    • AU4: [0, 1]. Brow Lowerer. Default: null.
      • Any of
        • number
        • null
    • AU5: [0, 1]. Upper Lid Raiser. Default: null.
      • Any of
        • number
        • null
    • AU6: [0, 1]. Cheek Raiser. Default: null.
      • Any of
        • number
        • null
    • AU7: [0, 1]. Lid Tightener. Default: null.
      • Any of
        • number
        • null
    • AU9: [0, 1]. Nose Wrinkler. Default: null.
      • Any of
        • number
        • null
    • AU10: [0, 1]. Upper Lip Raiser. Default: null.
      • Any of
        • number
        • null
    • AU12: [0, 1]. Lip Corner Puller. Default: null.
      • Any of
        • number
        • null
    • AU15: [0, 1]. Lip Corner Depressor. Default: null.
      • Any of
        • number
        • null
    • AU17: [0, 1]. Chin Raiser. Default: null.
      • Any of
        • number
        • null
    • AU20: [0, 1]. Lip Stretcher. Default: null.
      • Any of
        • number
        • null
    • AU23: [0, 1]. Lip Tightener. Default: null.
      • Any of
        • number
        • null
    • AU24: [0, 1]. Lip Pressor. Default: null.
      • Any of
        • number
        • null
    • AU25: [0, 1]. Lips Part. Default: null.
      • Any of
        • number
        • null
    • AU26: [0, 1]. Jaw Drop. Default: null.
      • Any of
        • number
        • null

Examples

{
    "face_action_units": [
        {
            "AU1": 0.5,
            "AU10": 0.5,
            "AU12": 0.5,
            "AU15": 0.5,
            "AU17": 0.5,
            "AU2": 0.5,
            "AU20": 0.5,
            "AU23": 0.5,
            "AU24": 0.5,
            "AU25": 0.5,
            "AU26": 0.5,
            "AU4": 0.5,
            "AU5": 0.5,
            "AU6": 0.5,
            "AU7": 0.5,
            "AU9": 0.5
        }
    ]
}

face/expression

Indicate the confidence level of each expression.

Properties

  • face_expression (array, required)

Definitions

  • Expression (object)
    • neutral: Confidence in [0, 1]. Default: null.
      • Any of
        • number
        • null
    • anger: Confidence in [0, 1]. Default: null.
      • Any of
        • number
        • null
    • disgust: Confidence in [0, 1]. Default: null.
      • Any of
        • number
        • null
    • fear: Confidence in [0, 1]. Default: null.
      • Any of
        • number
        • null
    • happy: Confidence in [0, 1]. Default: null.
      • Any of
        • number
        • null
    • sad: Confidence in [0, 1]. Default: null.
      • Any of
        • number
        • null
    • surprise: Confidence in [0, 1]. Default: null.
      • Any of
        • number
        • null

Examples

{
    "face_expression": [
        {
            "anger": 0.01,
            "disgust": 0.01,
            "fear": 0.01,
            "happy": 0.94,
            "neutral": 0.01,
            "sad": 0.01,
            "surprise": 0.01
        }
    ]
}

face/dense_landmarks

280-point dense facial landmarks (560 interleaved [x,y] floats).

Properties

  • face_dense_landmarks (array, required)

Definitions

  • DenseLandmarks (object)
    • landmarks (array, required): Length must be equal to 560.
      • Items (number)

Examples

{
    "face_dense_landmarks": [
        {
            "landmarks": "[100.0] * 560"
        }
    ]
}

face/feature

Per-face L2-normalized feature embeddings for recognition, clustering, or similarity search. Each inner list corresponds to one aligned face in face/landmarks order; vector dimension is algorithm-specific (typically 512 for ArcFace).

Properties

  • face_feature (array, required)
    • Items (array)
      • Items (number)

Examples

{
    "face_feature": [
        [
            0.0412,
            -0.0187,
            0.0934,
            0.0052,
            -0.0621
        ],
        [
            -0.0298,
            0.0745,
            0.0102,
            -0.0881,
            0.0356
        ]
    ]
}

face/mesh

468-point 3D face mesh landmarks in normalized coordinates.

Properties

Definitions

  • MeshLandmarks (object)
    • normalized_3d_landmarks (array, required): Length must be equal to 1404.
      • Items (number)

Examples

{
    "face_mesh": [
        {
            "normalized_3d_landmarks": "[0.5] * 1404"
        }
    ]
}

face/gaze_screen

Per-eye screen-space point-of-gaze coordinates in pixels and camera-space point-of-gaze coordinates in millimeters. Camera-space coordinates may be 2D or 3D depending on the algorithm; origin is at the camera center.

Properties

  • face_gaze_screen (array, required)

Definitions

  • GazeData (object): Gaze estimation result for a single face.
    • success (boolean, required): Whether gaze estimation succeeded for this face.
    • gaze_screen_px (required): Per-eye point-of-gaze coordinates in screen-space pixels. The origin is at the left top corner of the screen. The coordinate system is shown as Fig. 1. Refer to GazePoint.
    • gaze_camera_mm (required): Per-eye point-of-gaze coordinates in the camera coordinate system, measured in millimeters. The coordinate origin is located at the camera optical center. Depending on the gaze estimation algorithm, the point-of-gaze can be represented either as a 2D coordinate on the camera image plane or as a 3D point in camera space. The coordinate definitions for 2D and 3D gaze_camera_mm are illustrated in Fig. 2 and Fig. 3, respectively. Different algorithms may natively output in different coordinate frames; all values are normalized to the conventions documented here before being returned. Refer to GazePoint.
  • GazePoint (object): Point-of-gaze coordinates for two eyes in either screen-pixel or camera-millimeter space.
    • left_eye (array, required): Gaze-screen-px [x,y] or gaze-camera-mm [x,y(,z)]; 2D or 3D depending on algorithm; empty list if unavailable. Length must be between 0 and 3 (inclusive).
      • Items (number)
    • right_eye (array, required): Screen-px [x,y] or camera-mm [x,y(,z)]; 2D or 3D depending on algorithm; empty list if unavailable. Length must be between 0 and 3 (inclusive).
      • Items (number)
  • GazeScreen (object): Wrapper for per-face gaze data in face/gaze_screen schema.
Coordinate system for gaze_screen_px

Figure 1. Coordinate system for gaze_screen_px

Coordinate system for gaze_camera_mm(2D)

Figure 2. Coordinate system for gaze_camera_mm(2D)

Coordinate system for gaze_camera_mm(3D)

Figure 3. Coordinate system for gaze_camera_mm(3D)

Examples

{
    "face_gaze_screen": [
        {
            "gaze": {
                "gaze_camera_mm": {
                    "left_eye": [
                        155.0,
                        50.0,
                        25.0
                    ],
                    "right_eye": [
                        155.0,
                        50.0,
                        25.0
                    ]
                },
                "gaze_screen_px": {
                    "left_eye": [
                        960.0,
                        540.0
                    ],
                    "right_eye": [
                        960.0,
                        540.0
                    ]
                },
                "success": true
            }
        }
    ]
}

face/heart_rate

Heart rate (BPM) estimated from buffered face video frames.

Properties

  • face_heart_rate (required): Refer to HeartRate.

Definitions

  • HeartRate (object)
    • fps (number, required): Measured frames per second of the processing stream, averaged over a recent sliding window for stability.
    • wait_seconds (number, required): Rough estimate of remaining seconds until the next heart-rate update may be emitted. A value of 0.0 does not guarantee a result; use the presence of hr_bpm to determine whether a valid prediction is available.
    • hr_bpm: Final integrated heart-rate prediction in beats per minute. The combination strategy is algorithm-specific; this field is omitted entirely when the current payload does not carry a reliable estimate. Default: null.
      • Any of
        • number
        • null
    • roi_hr_bpm: Per-region heart-rate estimates keyed by the region identifier. Some regions may be absent from the mapping when no valid estimate can be produced for them, and the field as a whole is omitted for algorithms that do not expose ROI-level results. Default: null.
      • Any of
        • object: Can contain additional properties.
          • Additional properties (number)
        • null

Examples

{
    "face_heart_rate": {
        "fps": 30.0,
        "hr_bpm": 72.5,
        "roi_hr_bpm": {
            "skin_a_fixed_forehead": 72.5,
            "skin_b_adaptive_forehead": 72.5,
            "skin_c_connected_components": 72.5,
            "skin_legacy": 72.5
        },
        "wait_seconds": 0.0
    }
}
{
    "face_heart_rate": {
        "fps": 30.0,
        "wait_seconds": 5.2
    }
}

face/dimensional_affect

Continuous valence-arousal affect dimensions alongside discrete expressions and Action Units.

Properties

Definitions

  • DimensionalAffect (object)
    • valence (number, required): Valence dimension in continuous affect space. Positive = pleasant, negative = unpleasant.
    • arousal (number, required): Arousal dimension in continuous affect space. Positive = activated, negative = calm.

Examples

{
    "face_dimensional_affect": [
        {
            "arousal": 0.32,
            "valence": 0.85
        }
    ]
}

head/detection

Multi-person head bounding box detection results.

Properties

  • head_detection (array, required)

Definitions

  • HeadBBox (object)
    • xyxy (array, required): Length must be equal to 4.
      • Items (integer)
    • score (number, required)

Examples

{
    "head_detection": [
        {
            "score": 0.85,
            "xyxy": [
                100,
                200,
                300,
                400
            ]
        }
    ]
}

head/selection

Top-N head selection result (count + original indices), reordering head_detection.

Properties

Definitions

  • HeadSelection (object)
    • count (integer, required): Number of selected head detections.
    • selected_indices (array, required): Indices of selected detections in the original head_detection list, before sorting.
      • Items (integer)

Examples

{
    "head_detection": [
        {
            "score": 0.85,
            "xyxy": [
                100,
                200,
                300,
                400
            ]
        }
    ],
    "head_selection": {
        "count": 1,
        "selected_indices": [
            0
        ]
    }
}

head/gaze_point

Per-head 2D scene gaze target point with associated likelihood heatmap.

Properties

  • head_gaze_point (array, required)

Definitions

  • HeadGazePoint (object)
    • head_location_xyxy (array, required): Length must be equal to 4.
      • Items (integer)
    • gaze_point_px (array, required): Length must be equal to 2.
      • Items (number)
    • heatmap (array, required): 2D gaze likelihood heatmap over the scene. Runtime type: numpy.ndarray of float32, shape [image_height, image_width], values in [0, 1] probability range.
      • Items (array)
        • Items (number)

Examples

{
    "head_gaze_point": [
        {
            "gaze_point_px": [
                640.0,
                360.0
            ],
            "head_location_xyxy": [
                100,
                200,
                300,
                400
            ],
            "heatmap": "numpy.ndarray(shape=[H, W], dtype=float32) -- 2D [0,1] gaze likelihood heatmap"
        }
    ]
}

head/social_gaze

Dyadic social gaze relations between two detected people. Class set: share, mutual, single, miss, void.

Properties

Definitions

  • HeadSocialGaze (object)
    • principal: Left-side / primary person in dyadic interaction. Default: null.
    • associate: Right-side / secondary person in dyadic interaction. Default: null.
    • success (boolean): Whether at least two heads were detected for social gaze inference. Default: true.
  • SocialGazePerson (object)
    • head_location_xyxy (array, required): Length must be equal to 4.
      • Items (integer)
    • gaze_point_px (array, required): Length must be equal to 2.
      • Items (number)
    • heatmap (array, required): 2D gaze likelihood heatmap. Runtime type: numpy.ndarray of float32, shape [image_height, image_width], values in [0, 1] probability range.
      • Items (array)
        • Items (number)
    • social_gaze_id (integer, required): Integer class ID of the social gaze relation. Ordered mapping: 0=share, 1=mutual, 2=single, 3=miss, 4=void.
    • social_gaze_label (string, required): Human-readable social gaze relation label. Possible values: share, mutual, single, miss, void. Index of the value matches social_gaze_id.

Examples

{
    "head_social_gaze": {
        "associate": {
            "gaze_point_px": [
                200.0,
                300.0
            ],
            "head_location_xyxy": [
                600,
                200,
                800,
                400
            ],
            "heatmap": "numpy.ndarray(shape=[H, W], dtype=float32) -- 2D [0,1] gaze likelihood heatmap",
            "social_gaze_id": 1,
            "social_gaze_label": "mutual"
        },
        "principal": {
            "gaze_point_px": [
                800.0,
                300.0
            ],
            "head_location_xyxy": [
                100,
                200,
                300,
                400
            ],
            "heatmap": "numpy.ndarray(shape=[H, W], dtype=float32) -- 2D [0,1] gaze likelihood heatmap",
            "social_gaze_id": 1,
            "social_gaze_label": "mutual"
        },
        "success": true
    }
}
{
    "head_social_gaze": {
        "success": false
    }
}

Metadata

Release files for seetapsych-attributes 0.0.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for seetapsych-attributes 0.0.4
File Size Uploaded Provenance Actions
seetapsych_attributes-0.0.4.tar.gz 1.3 MB With provenance Details for seetapsych_attributes-0.0.4.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for seetapsych-attributes 0.0.4
File Interpreter ABI Platform Actions
seetapsych_attributes-0.0.4-py3-none-any.whl Python 3 none any Details for seetapsych_attributes-0.0.4-py3-none-any.whl

Total release size: 1.3 MB

Release files / seetapsych_attributes-0.0.4.tar.gz

Download URL seetapsych_attributes-0.0.4.tar.gz
Size 1.3 MB
Tags Source
SHA-256 checksum
How to use checksums
a251b8d23cb038a6cbece1cc2f88f63da8ad5126f119e2ce394bb9c5b34675cf
BLAKE2b-256 checksum
How to use checksums
f7bdfe491668e6274bf09e70988a35b0b7810cb3eea7d01f2861885a251c0684
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 17, 2026.

Transparency log

Release files / seetapsych_attributes-0.0.4-py3-none-any.whl

Download URL seetapsych_attributes-0.0.4-py3-none-any.whl
Size 27.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
cd67eff17bf97e176dc3a7dae1f5f53977b7b46449aae434d79348261d3669ba
BLAKE2b-256 checksum
How to use checksums
d15741c6bb24cdb96cb218ff6fd681c31fe0a466af2e392581fd74feaf934022
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 17, 2026.

Transparency log

Security for seetapsych-attributes 0.0.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for seetapsych_attributes-0.0.4.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 17, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page