myoktros 0.1.0

Myo EMG-based KT system for ROS

MyoKTROS

Myo EMG-based KT system for ROS.

• Installation
  • PIP
  • Build with Poetry
• Usage
  • keras Mode Preparation
  • knn Mode Preparation
  • Visualizing the State Machine
    • WebMachine
    • GraphMachine
• Myo
• Authors

Installation

PIP

pip install -U myoktros

Build with Poetry

Install Poetry first.

git clone https://github.com/Interactions-HSG/MyoKTROS.git && cd MyoKTROS
poetry install
poetry run myoktros

Usage

❯ myoktros -h
usage: myoktros [-h] [--mode {keras,knn}] [-a ADDRESS] [-d] [-m MAC] [--knn_samples KNN_SAMPLES] [--knn_periods KNN_PERIODS] [-p PORT]

Myo EMG-based KT system for ROS

options:
  -h, --help            show this help message and exit
  --mode {keras,knn}
                        mode to select (default: keras)
  -a ADDRESS, --address ADDRESS
                        the IP address for the ROS server (default: 127.0.0.1)
  -d, --debug           sets the log level to debug (default: False)
  -m MAC, --mac MAC     specify the mac address for Myo (default: None)
  --knn_samples KNN_SAMPLES
                        number of samples for the knn classifier (default: 3)
  --knn_periods KNN_PERIODS
                        number of sampling periods for the knn classifier (default: 10)
  -p PORT, --port PORT  the port for the ROS server (default: 8765)

keras Mode Preparation

Use a keras.Sequential model with sampling normalization to detect gestures.

You need to collect EMG data for gesutures and train a gesture model.

1. scripts/record_myo_data.py records EMG streams for one gesture to a CSV file in /assets. Repeat this step for N gestures.

❯ poetry run scripts/record_myo_data.py -h
usage: record_myo_data.py [-h] [--emg_mode {1,2,3}] [--mac MAC] [--seconds SECONDS] N

Record train data from EMG data stream via Myo

positional arguments:
  N                   the gesture to record (the enum value of myoktros.Gesture)

options:
  -h, --help          show this help message and exit
  --emg_mode {1,2,3}  set the myo.EMGMode (1: filtered/rectified, 2: filtered/unrectified, 3: unfiltered/unrectified) (default: 1)
  --mac MAC           the mac address for Myo (default: None)
  --seconds SECONDS   the duration to record in seconds (default: 30)

for example, to record EMG data for gesture 3:

❯ poetry run scripts/record_myo_data.py 3
2023-06-14 21:08:42,227 __main__ INFO: scanning for a Myo device...
2023-06-14 21:08:45,104 myoktros.myo_client INFO: connected to Myonnaise
2023-06-14 21:08:45,104 __main__ INFO: connected to a Myo
2023-06-14 21:08:45,108 myoktros.myo_client INFO: setting up the myo: Myonnaise
2023-06-14 21:08:45,133 myoktros.myo_client INFO: remaining battery: 91 %
2023-06-14 21:08:45,343 __main__ INFO: start recording BEND_WRIST data with SEND_FILT for 30 seconds
2023-06-14 21:08:47,345 myoktros.myo_client INFO: start notifying from Myonnaise
2023-06-14 21:09:17,545 myoktros.myo_client INFO: stopped notification from Myonnaise
2023-06-14 21:09:17,546 myoktros.myo_client INFO: sleep Myonnaise
2023-06-14 21:09:18,115 __main__ INFO: saved the recorded data to /Users/iomz/ghq/github.com/Interactions-HSG/MyoKTROS/data/SEND_FILT-BEND_WRIST-20230614210845.csv

2. scripts/build_keras_model.py builds a Keras.Sequential model based on the CSV files from 1.

❯ poetry run scripts/build_keras_model.py
WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
WARNING:absl:There is a known slowdown when using v2.11+ Keras optimizers on M1/M2 Macs. Falling back to the legacy Keras optimizer, i.e., `tf.keras.optimizers.legacy.Adam`.
Epoch 1/20
282/282 [==============================] - 0s 495us/step - loss: 0.3811
Epoch 2/20
282/282 [==============================] - 0s 495us/step - loss: 0.1141
Epoch 3/20
282/282 [==============================] - 0s 493us/step - loss: 0.0830
Epoch 4/20
282/282 [==============================] - 0s 484us/step - loss: 0.0630
Epoch 5/20
282/282 [==============================] - 0s 488us/step - loss: 0.0524
Epoch 6/20
282/282 [==============================] - 0s 483us/step - loss: 0.0459
Epoch 7/20
282/282 [==============================] - 0s 488us/step - loss: 0.0425
Epoch 8/20
282/282 [==============================] - 0s 486us/step - loss: 0.0380
Epoch 9/20
282/282 [==============================] - 0s 487us/step - loss: 0.0357
Epoch 10/20
282/282 [==============================] - 0s 497us/step - loss: 0.0336
Epoch 11/20
282/282 [==============================] - 0s 491us/step - loss: 0.0335
Epoch 12/20
282/282 [==============================] - 0s 491us/step - loss: 0.0310
Epoch 13/20
282/282 [==============================] - 0s 488us/step - loss: 0.0308
Epoch 14/20
282/282 [==============================] - 0s 491us/step - loss: 0.0279
Epoch 15/20
282/282 [==============================] - 0s 487us/step - loss: 0.0262
Epoch 16/20
282/282 [==============================] - 0s 494us/step - loss: 0.0259
Epoch 17/20
282/282 [==============================] - 0s 489us/step - loss: 0.0255
Epoch 18/20
282/282 [==============================] - 0s 489us/step - loss: 0.0243
Epoch 19/20
282/282 [==============================] - 0s 489us/step - loss: 0.0250
Epoch 20/20
282/282 [==============================] - 0s 483us/step - loss: 0.0234
poetry run scripts/build_keras_model.py  9.09s user 3.66s system 143% cpu 8.885 total

knn Mode Preparation

Use the k-NN classifier with sampling normalization to detect gestures.

First generate the classifier

poetry run scripts/build_knn_classifier.py

then run with --mode knn

poetry run myoktros --mode knn

Visualizing the State Machine

myoktros.Robot is the base robot class for Robots to be intereacted with, and a default finite-state machine is implemented with transitions.

transitions provides two methods to draw the diagram for the state machines.

WebMachine

transitions-gui implements WebMachine to produce a neat graph as a simple web service.

Run scripts/robot_web_machine.py (startup may take a few momemnt) and access http://localhost:8080?details=true on your browser.

You may need additional dependencies if not with poetry:

pip install transitions-gui tornado

GraphMachine

The state machine diagram can also be drawn using Graphviz with the dot layout engine by scripts/robot_graph_machine.py.

NOTE: pygraphviz cannot be installed straight for macOS, so not included in the poetry dependencies.

1. Install Graphviz: see here
2. Install python packages
   • for macOS

     pip install \
        --global-option=build_ext \
        --global-option="-I$(brew --prefix graphviz)/include/" \
        --global-option="-L$(brew --prefix graphviz)/lib/" \
        pygraphviz
     pip install graphviz
     

   • Otherwise

     pip install "transitions[diagrams]"
     

3. Generate the diagram (gets saved in assets/robot_state_diagram.png)

./scripts/assets/generate_robot_state_diagram.py

Myo

Myo Armbands are capable of streaming data as follows.

	EMGData	FVData	IMUData
Throughput	~200 S/s	~50 S/s	~50 S/s

Use scripts/speedometer.py to see it yourself.

❯ poetry run scripts/speedometer.py -h
usage: speedometer.py [-h] [--emg-mode {0,1,2,3}] [--imu-mode {0,1,2,3}] [--mac MAC] [--seconds SECONDS]

Measure the data stream throughput from Myo

options:
  -h, --help            show this help message and exit
  --emg-mode {0,1,2,3}  set the myo.types.EMGMode (0: disabled, 1: filtered/rectified, 2: filtered/unrectified, 3: unfiltered/unrectified) (default: 1)
  --imu-mode {0,1,2,3}  set the myo.types.IMUMode (0: disabled, 1: data, 2: event, 3: all) (default: 0)
  --mac MAC             the mac address for Myo (default: None)
  --seconds SECONDS     the duration to record in seconds (default: 10)

Authors

• Iori Mizutani (@iomz)
• Felix Wohlgemuth