gesto

Train and run gesture recognition models from Gesto Labeller datasets.

Point it at a project folder, pick a mode and a region, and it handles the rest — loading, training, versioned model storage, and live detection that matches how the data was captured.

pip install gesto

Two modes

Static needs far less data (every captured frame is a training sample) and predicts instantly with no warm-up. Reach for sequence only when two gestures share the same shape and differ by movement.

Five regions

The region must match how the project was captured — gesto checks the feature dimension and tells you if it doesn't.

Command line

# what's in this dataset?
gesto inspect ./gesto_projects/signs

# train
gesto train static hands_one ./gesto_projects/signs
gesto train sequence pose ./gesto_projects/jogging --seq-len 30

# detect (newest model by default)
gesto detect static hands_one
gesto detect sequence pose --source clip.mp4
gesto detect static hands_one --version 2

# what have I trained?
gesto list

Python

import gesto

run = gesto.train("./gesto_projects/signs", region="hands_one", mode="static")
gesto.detect("static", "hands_one")

Or drive a model yourself:

from gesto.detect import Predictor

predictor = Predictor.load("static", "hands_one")
vector = predictor.features(holistic_result)   # extract + normalize
probs = predictor.predict(vector)

Where models go

Everything lands under one artifacts/ folder, split by mode then region. Training never overwrites an earlier run — it versions:

artifacts/
    static/
        hands_one/          model.keras, labels.json
        hands_one_2/        the next run
        pose/
    sequence/
        pose/
        pose_2/

gesto detect static pose picks the newest version; --version 1 picks a specific one.

Matching your capture

Predictions are only correct when detection feeds the model the same kind of vector it trained on. gesto mirrors Gesto Labeller exactly:

• the same engine — MediaPipe Holistic, same confidence settings
• the same landmark order — including one-hand mode preferring the right hand and falling back to the left
• the same normalization — translation/scale-invariant, verified identical
• the same mirroring — webcam frames are flipped, video files are not

If you captured with Gesto's Normalise unchecked, pass --raw when training so detection knows to skip it.

Getting good results

• Balance your classes. Similar sample counts per class; gesto applies class weights but balanced data is better.
• Enough samples. ~20–30 static frames per class, or ~15–30 sequences. Small datasets automatically get a lighter model, since an oversized network on little data overfits and collapses to predicting one class.
• Consistent clip length for sequence mode — set "Max frames" in Gesto Labeller so every capture is the same length.

Run gesto inspect <project> to check all of this before training.

Requirements

Python 3.9+, TensorFlow, OpenCV, MediaPipe, NumPy — all installed with the package.

License

MIT