eyeGestures 4.0.0

Package for eye tracking algorithm allowing for development of gaze controlled computer interface

EYEGESTURES

EyeGestures is open source eyetracking software/library using native webcams and phone camers for achieving its goal. The aim of library is to bring accessibility of eye-tracking and eye-driven interfaces without requirement of obtaining expensive hardware.

Our Mission!

💜 Sponsors:

[!NOTE] Check JS port: EyeGesturesLite!

[!IMPORTANT]
EyeGestures is a fully volunteer-based project and exists thanks to your donations and support.

Donation

📢📢 If you are bussiness and would like to collaborate, reach us: [e-mail] 📢📢

We can provide custom comercial license and invoices if needed.

---

🔨 Projects build with EyeGestures:

• EyePilot
• EyePather - discontinued
• EyeFocus <- 4th best product on PH!
• Add your project! [e-mail] or PR

Subscribe and get access to our software:

---

⚓ Ports:

• JavaScript version: EyeGesturesLite!

💻 Install

python3 -m pip install eyeGestures

[!WARNING] some users report that mediapipe, scikit-learn or opencv is not installing together with eyegestures. To fix it, just install it with pip.

⚙️ Try

Tracker works best when your camera or laptop is at arm's length, similar to how you would typically use it. If you are further away, it may be less responsive for now - currently working on solving this issue.

python3 examples/simple_example_v2.py

python3 examples/simple_example.py [legacy tracker, will become obsolete]

🔧 Build your own:

Using EyeGesture Engine V3 - Faster smaller better:

from eyeGestures.utils import VideoCapture
from eyeGestures import EyeGestures_v3

# Initialize gesture engine and video capture
gestures = EyeGestures_v3()
cap = VideoCapture(0)
calibrate = True
screen_width = 500
screen_height= 500

# Process each frame
while True:
  ret, frame = cap.read()
  event, cevent = gestures.step(frame,
    calibrate,
    screen_width,
    screen_height,
    context="my_context")

  if event:
    cursor_x, cursor_y = event.point[0], event.point[1]
    fixation = event.fixation
    saccades = event.saccadess # saccadess movement detector
    # calibration_radius: radius for data collection during calibration

Using EyeGesture Engine V2 - [Legacy]:

from eyeGestures.utils import VideoCapture
from eyeGestures import EyeGestures_v2

# Initialize gesture engine and video capture
gestures = EyeGestures_v2()
cap = VideoCapture(0)  
calibrate = True
screen_width = 500
screen_height= 500

# Process each frame
while True:
  ret, frame = cap.read()
  event, cevent = gestures.step(frame,
    calibrate,
    screen_width,
    screen_height,
    context="my_context")

  if event:
    cursor_x, cursor_y = event.point[0], event.point[1]
    fixation = event.fixation
    # calibration_radius: radius for data collection during calibration

Customize [only V1/V2]:

You can customize your calibration points/map to fit your solutions. Simple copy snippet below, and place your calibration poitns on x,y planes from 0.0 to 1.0. It will be then automatically scaled to your display.

gestures = EyeGestures_v2()
gestures.uploadCalibrationMap([[0,0],[0,1],[1,0],[1,1]])

V2 is two stage tracker. It runs V1 under the hood but then uses it as feature extractor for V2 machine learning component, and combines both outputs to generate new gaze point. It is possible to control how much V1 affects V2 by:

gestures.setClassicImpact(N) # setting N = 2 is working best for my testing 

This makes that sample obtained from V2 is averaged with N times sample from V1 (same sample copied that many times). In outcome having V2 impacting output in 1/N+1 and V1 N/N+1.

It is also worth to know that you can enable hidden calibration for V1 (same calibration when using only V1, but now it is invisible to user):

gestures.enableCNCalib()

Using EyeGesture Engine V1 - Model-Based Approach [not recommended]:

from eyeGestures.utils import VideoCapture
from eyeGestures import EyeGestures_v1

# Initialize gesture engine with RoI parameters
gestures = EyeGestures_v1()

cap = VideoCapture(0)  
ret, frame = cap.read()
calibrate = True
screen_width = 500
screen_height= 500

# Obtain estimations from camera frames
event, cevent = gestures.estimate(
    frame,
    "main",
    calibrate,  # set calibration - switch to False to stop calibration
    screen_width,
    screen_height,
    0, 0, 0.8, 10
)

if event:
  cursor_x, cursor_y = event.point[0], event.point[1]
  fixation = event.fixation
  # calibration_radius: radius for data collection during calibration

Feel free to copy and paste the relevant code snippets for your project.

🔥 Web Demos:

• Main page
• EyeGesturesLite

rules of using

If you are building publicly available product, and have no commercial license, please mention us somewhere in your interface.

📇 Find us:

• discord
• twitter
• daily.dev
• email: [e-mail]

Troubleshooting:

1. some users report that mediapipe, scikit-learn or opencv is not installing together with eyegestures. To fix it, just install it with pip.

Missing feature or facing a bug?

Create new issue so we can tackle it!

💻 Contributors

💵 Support the project

We will be extremely grateful for your support: it helps to keep server running + fuels my brain with coffee.

Support project on Polar (in exchange we provide access to alphas versions!):