SPEED Pro v6

Overview

SPEED Pro v6 is a modular, high-performance Python 3.11+ framework re-engineered from the ground up for real-time eye-tracking data ingestion, computer vision inference, spatial mapping, and oculomotor event classification.

Designed according to Clean Architecture / Hexagonal Architecture (Ports and Adapters) and SOLID principles, SPEED Pro v6 completely decouples hardware protocols, mathematical algorithms, and storage backends. All domain contracts are immutably defined using Pydantic v2, static analysis is strictly validated with mypy --strict, and mathematical algorithms are pure and vectorized with NumPy and OpenCV for maximum execution speed.

Architectural Highlights

                       ┌─────────────────────────────────────────┐
                       │           Presentation / UI             │
                       └────────────────────┬────────────────────┘
                                            │
                       ┌────────────────────▼────────────────────┐
                       │          Application Pipeline           │
                       └────────────────────┬────────────────────┘
                                            │
        ┌───────────────────────────────────┴───────────────────────────────────┐
        │                          Core Domain Layer                            │
        │                                                                       │
        │  ┌──────────────────────┐  ┌─────────────────┐  ┌──────────────────┐  │
        │  │     Data Models      │  │ Core Interfaces │  │    Core Math     │  │
        │  │  - GazeDatum         │  │ - IEyeTracker   │  │  - Homography    │  │
        │  │  - FramePacket       │  │ - ICameraStream │  │  - I-VT Classifier│ │
        │  │  - Surface, AOI      │  │ - IDetector     │  │  - NSI Proximity │  │
        │  │  - BoundingBox       │  │ - ITelemetrySink│  │  - Switching Idx │  │
        │  └──────────────────────┘  └────────▲────────┘  └──────────────────┘  │
        └─────────────────────────────────────┼─────────────────────────────────┘
                                              │ Inversion of Control (Protocols)
        ┌─────────────────────────────────────┴─────────────────────────────────┐
        │                     Infrastructure & Adapters                         │
        │                                                                       │
        │  ┌──────────────────────┐  ┌─────────────────┐  ┌──────────────────┐  │
        │  │   Pupil Labs Neon    │  │ OpenCV / RTSP   │  │  PyArrow Parquet │  │
        │  │   Hardware Client    │  │  Camera Stream  │  │   Telemetry Sink │  │
        │  └──────────────────────┘  └─────────────────┘  └──────────────────┘  │
        └───────────────────────────────────────────────────────────────────────┘

• Clean / Hexagonal Architecture: The domain core (core.models, core.interfaces, core.math) has zero dependencies on infrastructure adapters, external hardware APIs, or UI toolkits.
• Strict Inversion of Control: Hardware capture devices and vision models are accessed via @runtime_checkable typing.Protocol interfaces.
• Data Integrity & Immutability: Domain models use ConfigDict(frozen=True) with Pydantic v2. Video frame buffers and segmentation masks are locked as read-only NumPy arrays to eliminate accidental cross-thread race conditions.
• Pure Vectorized Mathematics: Zero procedural loops in hot paths; all homography projections, I-VT classifications, and surface boundary calculations leverage vectorized NumPy broadcasting and BLAS routines.

Package Structure

speed_v6/
├── pyproject.toml                         # Project metadata, build system, mypy, ruff, pytest
├── LICENSE                                # MIT License
├── CITATION.cff                           # Academic citation metadata
├── README.md                              # Framework documentation
├── src/
│   └── speed_v6/
│       ├── __init__.py                    # speed_v6 package root
│       ├── py.typed                       # PEP 561 static typing marker
│       ├── core/
│       │   ├── models/                    # Immutable Pydantic v2 Data Contracts
│       │   │   ├── telemetry.py           # GazeDatum, FramePacket
│       │   │   ├── spatial.py             # Surface, DynamicAOI, HomographyMatrix
│       │   │   └── detection.py           # BoundingBox, DetectionResult
│       │   ├── interfaces/                # IoC Protocol Definitions
│       │   │   ├── camera.py              # ICameraStream
│       │   │   ├── eye_tracker.py         # IEyeTrackerStream
│       │   │   ├── detector.py            # IObjectDetector
│       │   │   └── storage.py             # ITelemetrySink
│       │   └── math/                      # Pure Vectorized Algorithms
│       │       ├── homography.py          # HomographyTransformer (RANSAC & batch projection)
│       │       ├── oculometrics.py        # IVTClassifier (Velocity-Threshold Identification)
│       │       └── nsi.py                 # NearestSurfaceIntersection & Switching Index
│       └── infrastructure/
│           └── hardware/
│               └── pupil_labs/
│                   └── client.py          # PupilLabsEyeTracker (Neon/Invisible adapter)
└── tests/
    ├── conftest.py                        # Pytest fixtures and async backend config
    ├── mocks/
    │   └── mock_eye_tracker.py            # MockEyeTrackerStream for high-freq simulation
    └── unit/
        ├── test_models.py                 # Pydantic contract validation tests
        ├── test_homography.py             # Homography projection & RANSAC unit tests
        ├── test_oculometrics.py           # I-VT classification & event extraction tests
        ├── test_nsi.py                    # NSI proximity and boundary distance tests
        └── test_adapters.py               # Hardware protocol & mock streaming tests

Core Algorithms & Mathematical Foundations

1. Homography & Perspective Transform (core.math.homography)

Estimates the projective matrix $H \in \mathbb{R}^{3 \times 3}$ mapping coordinates from camera image space to defined surface/screen space using RANSAC outlier rejection:

$$\begin{bmatrix} X \ Y \ W \end{bmatrix} = H \begin{bmatrix} x \ y \ 1 \end{bmatrix}, \quad x_{\text{surface}} = \frac{X}{W}, \quad y_{\text{surface}} = \frac{Y}{W}$$

• Fully vectorized batch projection for arrays of shape $(N, 2)$.
• Automatic root-mean-square reprojection error computation on inliers.
• Singular / near-zero $W$ denominator guards preventing zero-division errors.
• Ray-casting point-in-polygon verification.

2. Oculomotor I-VT Classifier (core.math.oculometrics)

Pure NumPy implementation of the Velocity-Threshold Identification (I-VT) algorithm:

$$\Delta t_i = (t_i - t_{i-1}) \cdot 10^{-9} \text{ s}$$

$$\Delta d_i = \sqrt{(x_i - x_{i-1})^2 + (y_i - y_{i-1})^2}$$

$$v_i = \frac{\Delta d_i}{\Delta t_i} \quad [\text{units/s or deg/s}]$$

• Classifies samples into FIXATION ($v_i < \theta$), SACCADE ($v_i \ge \theta$), BLINK, and UNKNOWN (low confidence).
• Aggregates contiguous fixation samples into FixationEvent objects with start/end timestamps, duration (ms), centroid coordinates $(\bar{x}, \bar{y})$, and RMS spatial dispersion.

3. Nearest Surface Intersection (NSI) (core.math.nsi)

Calculates the spatial proximity of gaze points to defined 2D polygonal surfaces:

• Vectorized point-to-polygon-centroid Euclidean distance $|\mathbf{p} - \mathbf{c}|_2$.
• Vectorized point-to-segment orthogonal projection across all edges of a polygon:

$$\mathbf{q} = \mathbf{a} + \text{clamp}\left(\frac{(\mathbf{p} - \mathbf{a}) \cdot (\mathbf{b} - \mathbf{a})}{|\mathbf{b} - \mathbf{a}|^2}, 0, 1\right) (\mathbf{b} - \mathbf{a})$$

• Ranks surfaces to identify the closest target.
• Computes the Normalized Switching Index (NSI) between Areas of Interest:

$$\text{NSI} = \frac{K - 1}{L_{\text{in}} - 1}$$

where $K$ is the number of transitions between consecutive distinct AOIs and $L_{\text{in}}$ is the total number of valid AOI gaze samples.

Installation

Prerequisites

• Python >= 3.11 (tested on Python 3.11 and 3.12)
• Recommended: uv package manager for high-speed deterministic builds.

Quick Setup with uv

# Clone repository
git clone https://github.com/danielelozzi/SPEED-pro.git
cd SPEED-pro/speed_v6

# Create virtual environment with Python 3.12
uv venv --python 3.12 .venv
source .venv/bin/activate

# Install in editable mode with development dependencies
uv pip install -e ".[dev]"

Setup with standard pip

cd SPEED-pro/speed_v6
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

Quickstart & Code Examples

1. High-Frequency Gaze Streaming with Mock Simulator

import asyncio
from tests.mocks.mock_eye_tracker import MockEyeTrackerStream

async def main() -> None:
    # Instantiate 120 Hz simulator
    tracker = MockEyeTrackerStream(frequency_hz=120.0, max_samples=20)
    await tracker.connect()

    print(f"Connected: {tracker.is_connected}")

    async for datum in tracker.stream_telemetry():
        print(
            f"Timestamp: {datum.timestamp_unix_ns} | "
            f"Gaze: ({datum.norm_x:.4f}, {datum.norm_y:.4f}) | "
            f"Confidence: {datum.confidence:.2f} | "
            f"Blink: {datum.is_blink}"
        )

    await tracker.disconnect()

if __name__ == "__main__":
    asyncio.run(main())

2. Oculomotor I-VT Classification

import numpy as np
from speed_v6.core.math.oculometrics import IVTClassifier

# Synthetic timestamps (100 Hz = 10 ms interval)
timestamps_ns = np.array([i * 10_000_000 for i in range(100)], dtype=np.int64)

# 70 stationary samples (fixation) followed by 30 leaping samples (saccade)
coords = np.zeros((100, 2))
coords[:70] = [0.3, 0.3]
coords[70:] = np.linspace([0.3, 0.3], [0.8, 0.8], 30)

# Classify with velocity threshold of 2.0 norm_units/s
labels = IVTClassifier.classify(
    timestamps_ns=timestamps_ns,
    coords=coords,
    velocity_threshold=2.0,
)

# Extract structured fixation events (minimum duration 100 ms)
fixations = IVTClassifier.extract_fixations(
    timestamps_ns=timestamps_ns,
    coords=coords,
    classifications=labels,
    min_duration_ms=100.0,
)

for fix in fixations:
    print(
        f"Fixation at ({fix.centroid_x:.3f}, {fix.centroid_y:.3f}) | "
        f"Duration: {fix.duration_ms:.1f} ms | "
        f"Samples: {fix.sample_count} | "
        f"Dispersion: {fix.dispersion:.5f}"
    )

3. Perspective Projection with Homography

import numpy as np
from speed_v6.core.math.homography import HomographyTransformer

# 4 corner correspondences (camera frame -> screen frame)
src = np.array([[0.1, 0.1], [0.9, 0.1], [0.85, 0.85], [0.15, 0.9]])
dst = np.array([[0.0, 0.0], [1.0, 0.0], [1.0, 1.0], [0.0, 1.0]])

# Estimate 3x3 projective matrix using RANSAC
h_matrix, mask, rms_err = HomographyTransformer.find_homography(src, dst)
print(f"Reprojection RMS Error: {rms_err:.4f}")

# Project a normalized gaze coordinate from camera space to screen space
screen_x, screen_y = HomographyTransformer.project_gaze(0.5, 0.5, h_matrix)
print(f"Projected Screen Gaze: ({screen_x:.4f}, {screen_y:.4f})")

4. Nearest Surface Intersection (NSI)

from speed_v6.core.models.spatial import Surface
from speed_v6.core.math.nsi import NearestSurfaceIntersection

cockpit_display = Surface(
    id="display_1",
    name="Primary Flight Display",
    vertices=((0.1, 0.1), (0.4, 0.1), (0.4, 0.6), (0.1, 0.6)),
)

hud_display = Surface(
    id="display_2",
    name="Head-Up Display",
    vertices=((0.6, 0.1), (0.9, 0.1), (0.9, 0.6), (0.6, 0.6)),
)

surfaces = [cockpit_display, hud_display]
gaze_point = (0.55, 0.3)

nearest = NearestSurfaceIntersection.find_nearest_surface(gaze_point, surfaces)
if nearest:
    print(f"Nearest Surface: {nearest.surface_name}")
    print(f"Inside: {nearest.is_inside}")
    print(f"Distance to boundary: {nearest.distance_to_boundary:.4f}")

Development & Quality Assurance

Run Unit Tests

.venv/bin/pytest -v

Run Strict Type Checking (mypy)

.venv/bin/mypy src/ tests/

Run Linter & Formatter (ruff)

# Check code style and imports
.venv/bin/ruff check src/ tests/

# Automatically format code
.venv/bin/ruff format src/ tests/

Roadmap

• Phase 0: Scaffolding & Core Domain
  • Pydantic v2 data contracts (GazeDatum, FramePacket, Surface, DynamicAOI, BoundingBox, DetectionResult).
  • Inversion of Control protocols (ICameraStream, IEyeTrackerStream, IObjectDetector, ITelemetrySink).
  • Vectorized math: Homography RANSAC, I-VT Classifier, Nearest Surface Intersection.
• Phase 1: Ingestion Hardware Skeleton
  • Pupil Labs Realtime API Neon/Invisible client adapter.
  • High-frequency mock eye-tracker simulator.
  • Unit test suite (pytest, mypy --strict, ruff).
• Phase 2: Computer Vision Pipeline (YOLOv8 / RT-DETR object detection, dynamic AOI tracking, ArUco/AprilTag surface detection).
• Phase 3: Real-time Stream Orchestration (Multi-threaded producer-consumer ring buffer, zero-copy IPC shared memory, LSL bridge).
• Phase 4: Persistence & Storage Sinks (Apache Arrow / Parquet high-throughput telemetry sink, BIDS export).
• Phase 5: Presentation & Interactive UI (PySide6 / WebGL dashboard, real-time gaze visualization, timeline AOI editor).

Citation

If you use SPEED or SPEED Pro in your research or academic publications, please cite:

@article{lozzi2024speed,
  title={SPEED: A Graphical User Interface Software for Processing Eye Tracking Data},
  author={Lozzi, Daniele and Di Pompeo, Ilaria and Marcaccio, Martina and Ademaj, Matias and Migliore, Simone and Curcio, Gianluca},
  journal={Software},
  volume={6},
  number={2},
  pages={35},
  year={2024},
  publisher={MDPI},
  doi={10.3390/software6020035},
  url={https://www.mdpi.com/2673-4087/6/2/35}
}

License

This project is licensed under the MIT License - see the LICENSE file for details.

Copyright (c) 2024-2026 Daniele Lozzi, Laboratorio di Scienze Cognitive e del Comportamento.