Skip to main content

Myo EMG-based KT system for ROS

Project description

MyoKTROS

build codecov python versions pypi version license

Myo EMG-based KT system with, but not limited to, ROS.

MyoKTROS directly communicates with a Myo Armband via dl-myo.

Installation

PIP

pip install -U myoktros

Build and install with Poetry

Install Poetry first.

git clone https://github.com/Interactions-HSG/MyoKTROS.git && cd MyoKTROS
poetry install
poetry build

Usage

❯ myoktros -h
usage: myoktros [-h] [-c CONFIG] {run,calibrate,test} ...

Myo EMG-based KT system for ROS

options:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        path to the config file (config.ini)

commands:
  {run,calibrate,test}

Configuration

The config.ini should be configured. Find the file in this repository for an actual example.

MyoKTROS initializes the avaialble gestures by reading this config file.

[myoktros]
gestures = # list of gestures

[myoktros.trigger_map]
# mapping of the gestures to the `RobotModel` transitions triggers

[myoktros.robot]
driver = # driver to connect to the robot

[myoktros.robot.xarm_rosws]
# driver-specific configurations

Gesture Model Calibration

You need to record EMG data for training a gesture model (keras, knn, or svm).

The myoktros calibrate command lets you record the EMG data and save them as CSV files, and then train a model (default: tensorflow.keras.Sequential).

There are model-specific parameters prefixed by the model-type, and the parameters not meant for the chosen model are ignored.

If the model-specific parameters are configured for training, they also need to be passed to the myoktros run and myoktros test commands in order to load the trained model.

Otherwise, the model trained by the default parameters is loaded.

❯ myoktros calibrate -h
usage: myoktros calibrate [-h] [--arm-dominance {left,right}] [--data DATA] [--duration DURATION] [--emg-mode EMG_MODE] [-g GESTURE]
                          [--knn-algorithm {auto,brute,ball_tree,kd_tree}] [--knn-k KNN_K] [--knn-metric {minkowski,euclidean,manhattan}] [--mac MAC]
                          [--model-type {keras,knn,svm}] [--n-samples N_SAMPLES] [--svm-c SVM_C] [--svm-degree SVM_DEGREE] [--svm-gamma {scale,auto}]
                          [--svm-kernel {linear,poly,rbf,sigmoid}] [-v] [--only-record | --only-train]

Calibrate the gesture model by recoding the user's EMG

options:
  -h, --help            show this help message and exit
  --arm-dominance {left,right}
                        left/right arm wearing the Myo (default: right)
  --data DATA           path to the data directory to save recorded data (default: /Users/iomz/ghq/github.com/Interactions-HSG/MyoKTROS/data)
  --duration DURATION   seconds to record each gesture for recoding (default: 30)
  --emg-mode EMG_MODE   set the myo.types.EMGMode to calibrate with (1: filtered/rectified, 2: filtered/unrectified, 3: unfiltered/unrectified) (default: 1)
  -g GESTURE, --gesture GESTURE
                        if specified, only record a specific gesture (default: all)
  --knn-algorithm {auto,brute,ball_tree,kd_tree}
                        algorithm for fitting the knn model (default: auto)
  --knn-k KNN_K         k for fitting the knn model (default: 5)
  --knn-metric {minkowski,euclidean,manhattan}
                        distance metric for fitting the knn model (default: minkowski)
  --mac MAC             specify the mac address for Myo (default: None)
  --model-type {keras,knn,svm}
                        model type to calibrate (default: keras)
  --n-samples N_SAMPLES
                        number of samples to detect a gesture (default: 50)
  --svm-c SVM_C         regularization parameter C for fitting the svm model (must be strictly positive) (default: 1.0)
  --svm-degree SVM_DEGREE
                        degree of polynomial function for fitting the svm model with 'poly' kernel (default: 3)
  --svm-gamma {scale,auto}
                        kernel coefficient for fitting the svm model (default: scale)
  --svm-kernel {linear,poly,rbf,sigmoid}
                        kernel type for fitting the svm model (default: poly)
  -v, --verbose         sets the log level to debug (default: False)
  --only-record         only record EMG data without training a model (default: False)
  --only-train          only train a model without recording EMG data (default: False)

Connecting MyoKTROS to a robot

MyoKTROS currently supports the following robots.

UFACTORY

xarm_ros2

xarm_ros2 includes the ROS2 packages for xArm/Lite6 robot series.

The ROS nodes to communicate with xArm/Lite6 robots can be deployed by the included docker-compose.yml. MyoKTROS invokes the xarm_api services via internal WebSocket messages whose payloads are analogous to the SRV files for the corresponding services.

Note that, the ros2-xarm container attempts to connect to the robot accessible at the IP address specified by the environmental variable ROBOT_IP.

Depending on the model, you need to modify the launch command. For example:

  • for Lite6 robots
ros2 launch xarm_api lite6_driver.launch.py robot_ip:="${ROBOT_IP}"
  • for xArm7 robots
ros2 launch xarm_api xarm7_driver.launch.py robot_ip:="${ROBOT_IP}"

Finally, set myoktros.robot.driver to xarm_rosws and configure the parameters.

[myoktros.robot.xarm_rosws]
model = lite6 # the robot model
ip = 127.0.0.1 # the IP address for the ROS service client
port = 8765 # the port that the ROS service client's websocket interface is listening at
WebSocket API

UFACTORY's proprietary WebSocket API: https://github.com/xArm-Developer/ufactory_docs/blob/main/websocketapi/websocketapi.md

With this API driver, MyoKTROS can operate without ROS.

Set myoktros.robot.driver to ufactory_ws and configure the parameters.

[myoktros.robot.ufactory_ws]
model = lite6 # the robot model
ip = 10.0.0.6 # the IP address for the robot controller

Visualizing the State Machine

myoktros.Robot is the base robot class for Robots to be intereacted with, and a default finite-state machine is implemented with transitions.

transitions provides two methods to draw the diagram for the state machines.

WebMachine

transitions-gui implements WebMachine to produce a neat graph as a simple web service.

Run scripts/robot_web_machine.py (startup may take a few momemnt) and access http://localhost:8080?details=true on your browser.

You may need additional dependencies if not with poetry:

pip install transitions-gui tornado

robot_web_machine

GraphMachine

The state machine diagram can also be drawn using Graphviz with the dot layout engine by scripts/robot_graph_machine.py.

NOTE: pygraphviz cannot be installed straight for macOS, so not included in the poetry dependencies.

  1. Install Graphviz: see here
  2. Install python packages
    • for macOS
      pip install \
         --global-option=build_ext \
         --global-option="-I$(brew --prefix graphviz)/include/" \
         --global-option="-L$(brew --prefix graphviz)/lib/" \
         pygraphviz
      pip install graphviz
      
    • Otherwise
      pip install "transitions[diagrams]"
      
  3. Generate the diagram (gets saved in assets/robot_state_diagram.png)
./scripts/assets/generate_robot_state_diagram.py

robot_graph_machine

Gestures

Any gestures can be registered in config.ini, but these are some common examples.

REST

REST-01

GRAB

GRAB-01 GRAB-02

STRETCH_FINGERS

STRETCH_FINGERS-01 STRETCH_FINGERS-02

EXTENSION

EXTENSION-01

FLEXION

FLEXION-01

HORN

HORN-01 HORN-02

FLEMING

FLEMING-01 FLEMING-02

THUMBS_UP

THUMBS_UP-01

SHOOT

SHOOT-01 SHOOT-02

TENNET

TENNET-01

Myo

Myo Armbands are capable of streaming data as follows.

EMGData FVData IMUData
Throughput ~200 S/s ~50 S/s ~50 S/s

Use scripts/speedometer.py to see it yourself.

❯ ./scripts/speedometer.py -h
usage: speedometer.py [-h] [--emg-mode {0,1,2,3}] [--imu-mode {0,1,2,3}] [--mac MAC] [--seconds SECONDS]

Measure the data stream throughput from Myo

options:
  -h, --help            show this help message and exit
  --emg-mode {0,1,2,3}  set the myo.types.EMGMode (0: disabled, 1: filtered/rectified, 2: filtered/unrectified, 3: unfiltered/unrectified) (default: 1)
  --imu-mode {0,1,2,3}  set the myo.types.IMUMode (0: disabled, 1: data, 2: event, 3: all) (default: 0)
  --mac MAC             the mac address for Myo (default: None)
  --seconds SECONDS     the duration to record in seconds (default: 10)

Authors

  • Iori Mizutani (@iomz)
  • Felix Wohlgemuth

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

myoktros-0.3.0.tar.gz (20.8 kB view details)

Uploaded Source

Built Distribution

myoktros-0.3.0-py3-none-any.whl (18.3 kB view details)

Uploaded Python 3

File details

Details for the file myoktros-0.3.0.tar.gz.

File metadata

  • Download URL: myoktros-0.3.0.tar.gz
  • Upload date:
  • Size: 20.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.1 CPython/3.11.4

File hashes

Hashes for myoktros-0.3.0.tar.gz
Algorithm Hash digest
SHA256 221a799c44130601eb4cc6d5c5b3cc2c520a0f9b1c9821324fe418e595bc6e79
MD5 381a856166b435a3767104b02a149fea
BLAKE2b-256 fd6c0163e937aef6ad5bf17d72973626606205375ea772e53c528f6be1cfd6eb

See more details on using hashes here.

File details

Details for the file myoktros-0.3.0-py3-none-any.whl.

File metadata

  • Download URL: myoktros-0.3.0-py3-none-any.whl
  • Upload date:
  • Size: 18.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.1 CPython/3.11.4

File hashes

Hashes for myoktros-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ce719e5997ae0f934a7da2967bda82e3e9c4ec6fe8d3774fadf70fed88df84b6
MD5 adba01b65521c0e80792bb2ab7c336ad
BLAKE2b-256 67bcf04285a7fc8d7aee4609486a34ec981679e1ea8c8b8b5d86b15f611ecea1

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page