sightsprite

Real-time machine vision with voice alerts.

Classification with voice alerts. Provides all the infrastructure needed to set things up, from data collection to real-time inference.

Aims for sightsprite:

• capture data (capture.py)
• label images (training.py)
• train model (training.py)
• Run inference (inference.py)
• Real-time inference with voice alerts (inference.py)

Uses for sightsprite

Could be used for any classification task where yuou want voice alerts when certain types of visual inputs are detected (e.g., dog is on furniture). Main initial use case is extremely important: detecting when I'm falling asleep on the couch at night and it yells at me to go to bed.

Plan

Once following steps working I'll consider sightsprite to be in alpha stage:

1. Get capture.py in slightly better shape
   • Add ability to change width/height instead of default width x height
   • fix up the weird Qt warnings in Linux
   • Make some simple tests
   • Capture some more data
2. Train on sleep data (use deepglue), build out training.py module.
   • Label sleep data using ImageLabeler
   • Train network using deepglue tools within sandbox.
3. Once training infrastructure set up, build out inference.py
   • Real-time inference
   • Voice alerts to nudge you.
4. Build some docs to explain how it all works...

To do

• Change build back end to uv (update pyproject.toml).
• Capture a little bit more, but not too much, just enough to reignite.
• Set up minimal training pipeline.
• Inference w/o voice alerts (want high F1 for sleep label)
• Inference w/voice alerts.
• Test out in the wild.
• Regroup, go back and add more bells and whistles.

Notes etc

Any images skipped during labeling will appear first when you restart ImageLabeler