myoktros 0.3.0

Myo EMG-based KT system for ROS

MyoKTROS

Myo EMG-based KT system with, but not limited to, ROS.

MyoKTROS directly communicates with a Myo Armband via dl-myo.

• Installation
  • PIP
  • Build and install with Poetry
• Usage
  • Configuration
  • Gesture Model Calibration
  • Connecting MyoKTROS to a robot
    • UFACTORY
      • xarm_ros2
      • WebSocket API
• Visualizing the State Machine
  • WebMachine
  • GraphMachine
• Gestures
  • REST
  • GRAB
  • STRETCH_FINGERS
  • EXTENSION
  • FLEXION
  • HORN
  • FLEMING
  • THUMBS_UP
  • SHOOT
  • TENNET
• Myo
• Authors

Installation

PIP

pip install -U myoktros

Build and install with Poetry

Install Poetry first.

git clone https://github.com/Interactions-HSG/MyoKTROS.git && cd MyoKTROS
poetry install
poetry build

Usage

❯ myoktros -h
usage: myoktros [-h] [-c CONFIG] {run,calibrate,test} ...

Myo EMG-based KT system for ROS

options:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        path to the config file (config.ini)

commands:
  {run,calibrate,test}

Configuration

The config.ini should be configured. Find the file in this repository for an actual example.

MyoKTROS initializes the avaialble gestures by reading this config file.

[myoktros]
gestures = # list of gestures

[myoktros.trigger_map]
# mapping of the gestures to the `RobotModel` transitions triggers

[myoktros.robot]
driver = # driver to connect to the robot

[myoktros.robot.xarm_rosws]
# driver-specific configurations

Gesture Model Calibration

You need to record EMG data for training a gesture model (keras, knn, or svm).

The myoktros calibrate command lets you record the EMG data and save them as CSV files, and then train a model (default: tensorflow.keras.Sequential).

There are model-specific parameters prefixed by the model-type, and the parameters not meant for the chosen model are ignored.

If the model-specific parameters are configured for training, they also need to be passed to the myoktros run and myoktros test commands in order to load the trained model.

Otherwise, the model trained by the default parameters is loaded.

❯ myoktros calibrate -h
usage: myoktros calibrate [-h] [--arm-dominance {left,right}] [--data DATA] [--duration DURATION] [--emg-mode EMG_MODE] [-g GESTURE]
                          [--knn-algorithm {auto,brute,ball_tree,kd_tree}] [--knn-k KNN_K] [--knn-metric {minkowski,euclidean,manhattan}] [--mac MAC]
                          [--model-type {keras,knn,svm}] [--n-samples N_SAMPLES] [--svm-c SVM_C] [--svm-degree SVM_DEGREE] [--svm-gamma {scale,auto}]
                          [--svm-kernel {linear,poly,rbf,sigmoid}] [-v] [--only-record | --only-train]

Calibrate the gesture model by recoding the user's EMG

options:
  -h, --help            show this help message and exit
  --arm-dominance {left,right}
                        left/right arm wearing the Myo (default: right)
  --data DATA           path to the data directory to save recorded data (default: /Users/iomz/ghq/github.com/Interactions-HSG/MyoKTROS/data)
  --duration DURATION   seconds to record each gesture for recoding (default: 30)
  --emg-mode EMG_MODE   set the myo.types.EMGMode to calibrate with (1: filtered/rectified, 2: filtered/unrectified, 3: unfiltered/unrectified) (default: 1)
  -g GESTURE, --gesture GESTURE
                        if specified, only record a specific gesture (default: all)
  --knn-algorithm {auto,brute,ball_tree,kd_tree}
                        algorithm for fitting the knn model (default: auto)
  --knn-k KNN_K         k for fitting the knn model (default: 5)
  --knn-metric {minkowski,euclidean,manhattan}
                        distance metric for fitting the knn model (default: minkowski)
  --mac MAC             specify the mac address for Myo (default: None)
  --model-type {keras,knn,svm}
                        model type to calibrate (default: keras)
  --n-samples N_SAMPLES
                        number of samples to detect a gesture (default: 50)
  --svm-c SVM_C         regularization parameter C for fitting the svm model (must be strictly positive) (default: 1.0)
  --svm-degree SVM_DEGREE
                        degree of polynomial function for fitting the svm model with 'poly' kernel (default: 3)
  --svm-gamma {scale,auto}
                        kernel coefficient for fitting the svm model (default: scale)
  --svm-kernel {linear,poly,rbf,sigmoid}
                        kernel type for fitting the svm model (default: poly)
  -v, --verbose         sets the log level to debug (default: False)
  --only-record         only record EMG data without training a model (default: False)
  --only-train          only train a model without recording EMG data (default: False)

Connecting MyoKTROS to a robot

MyoKTROS currently supports the following robots.

UFACTORY

• xArm series: https://www.ufactory.cc/xarm-collaborative-robot/
• LITE 6: https://www.ufactory.cc/lite-6-collaborative-robot/

xarm_ros2

xarm_ros2 includes the ROS2 packages for xArm/Lite6 robot series.

The ROS nodes to communicate with xArm/Lite6 robots can be deployed by the included docker-compose.yml. MyoKTROS invokes the xarm_api services via internal WebSocket messages whose payloads are analogous to the SRV files for the corresponding services.

Note that, the ros2-xarm container attempts to connect to the robot accessible at the IP address specified by the environmental variable ROBOT_IP.

Depending on the model, you need to modify the launch command. For example:

• for Lite6 robots

ros2 launch xarm_api lite6_driver.launch.py robot_ip:="${ROBOT_IP}"

• for xArm7 robots

ros2 launch xarm_api xarm7_driver.launch.py robot_ip:="${ROBOT_IP}"

Finally, set myoktros.robot.driver to xarm_rosws and configure the parameters.

[myoktros.robot.xarm_rosws]
model = lite6 # the robot model
ip = 127.0.0.1 # the IP address for the ROS service client
port = 8765 # the port that the ROS service client's websocket interface is listening at

WebSocket API

UFACTORY's proprietary WebSocket API: https://github.com/xArm-Developer/ufactory_docs/blob/main/websocketapi/websocketapi.md

With this API driver, MyoKTROS can operate without ROS.

Set myoktros.robot.driver to ufactory_ws and configure the parameters.

[myoktros.robot.ufactory_ws]
model = lite6 # the robot model
ip = 10.0.0.6 # the IP address for the robot controller

Visualizing the State Machine

myoktros.Robot is the base robot class for Robots to be intereacted with, and a default finite-state machine is implemented with transitions.

transitions provides two methods to draw the diagram for the state machines.

WebMachine

transitions-gui implements WebMachine to produce a neat graph as a simple web service.

Run scripts/robot_web_machine.py (startup may take a few momemnt) and access http://localhost:8080?details=true on your browser.

You may need additional dependencies if not with poetry:

pip install transitions-gui tornado

GraphMachine

The state machine diagram can also be drawn using Graphviz with the dot layout engine by scripts/robot_graph_machine.py.

NOTE: pygraphviz cannot be installed straight for macOS, so not included in the poetry dependencies.

1. Install Graphviz: see here
2. Install python packages
   • for macOS

     pip install \
        --global-option=build_ext \
        --global-option="-I$(brew --prefix graphviz)/include/" \
        --global-option="-L$(brew --prefix graphviz)/lib/" \
        pygraphviz
     pip install graphviz
     

   • Otherwise

     pip install "transitions[diagrams]"
     

3. Generate the diagram (gets saved in assets/robot_state_diagram.png)

./scripts/assets/generate_robot_state_diagram.py

Gestures

Any gestures can be registered in config.ini, but these are some common examples.

REST

GRAB

STRETCH_FINGERS

EXTENSION

FLEXION

HORN

FLEMING

THUMBS_UP

SHOOT

TENNET

Myo

Myo Armbands are capable of streaming data as follows.

	EMGData	FVData	IMUData
Throughput	~200 S/s	~50 S/s	~50 S/s

Use scripts/speedometer.py to see it yourself.

❯ ./scripts/speedometer.py -h
usage: speedometer.py [-h] [--emg-mode {0,1,2,3}] [--imu-mode {0,1,2,3}] [--mac MAC] [--seconds SECONDS]

Measure the data stream throughput from Myo

options:
  -h, --help            show this help message and exit
  --emg-mode {0,1,2,3}  set the myo.types.EMGMode (0: disabled, 1: filtered/rectified, 2: filtered/unrectified, 3: unfiltered/unrectified) (default: 1)
  --imu-mode {0,1,2,3}  set the myo.types.IMUMode (0: disabled, 1: data, 2: event, 3: all) (default: 0)
  --mac MAC             the mac address for Myo (default: None)
  --seconds SECONDS     the duration to record in seconds (default: 10)

Authors

• Iori Mizutani (@iomz)
• Felix Wohlgemuth