Myo EMG-based KT system for ROS
Project description
MyoKTROS
Myo EMG-based KT system with, but not limited to, ROS.
MyoKTROS directly communicates with a Myo Armband via dl-myo.
Installation
PIP
pip install -U myoktros
Build and install with Poetry
Install Poetry first.
git clone https://github.com/Interactions-HSG/MyoKTROS.git && cd MyoKTROS
poetry install
poetry build
Usage
❯ myoktros -h
usage: myoktros [-h] [-c CONFIG] {run,calibrate,test} ...
Myo EMG-based KT system for ROS
options:
-h, --help show this help message and exit
-c CONFIG, --config CONFIG
path to the config file (config.ini)
commands:
{run,calibrate,test}
Configuration
The config.ini
should be configured. Find the file in this repository for an actual example.
MyoKTROS initializes the avaialble gestures by reading this config file.
[myoktros]
gestures = # list of gestures
[myoktros.trigger_map]
# mapping of the gestures to the `RobotModel` transitions triggers
[myoktros.robot]
driver = # driver to connect to the robot
[myoktros.robot.xarm_rosws]
# driver-specific configurations
Gesture Model Calibration
You need to record EMG data for training a gesture model (keras
, knn
, or svm
).
The myoktros calibrate
command lets you record the EMG data and save them as CSV files, and then train a model (default: tensorflow.keras.Sequential
).
There are model-specific parameters prefixed by the model-type
, and the parameters not meant for the chosen model are ignored.
If the model-specific parameters are configured for training, they also need to be passed to the myoktros run
and myoktros test
commands in order to load the trained model.
Otherwise, the model trained by the default parameters is loaded.
❯ myoktros calibrate -h
usage: myoktros calibrate [-h] [--arm-dominance {left,right}] [--data DATA] [--duration DURATION] [--emg-mode EMG_MODE] [-g GESTURE]
[--knn-algorithm {auto,brute,ball_tree,kd_tree}] [--knn-k KNN_K] [--knn-metric {minkowski,euclidean,manhattan}] [--mac MAC]
[--model-type {keras,knn,svm}] [--n-samples N_SAMPLES] [--svm-c SVM_C] [--svm-degree SVM_DEGREE] [--svm-gamma {scale,auto}]
[--svm-kernel {linear,poly,rbf,sigmoid}] [-v] [--only-record | --only-train]
Calibrate the gesture model by recoding the user's EMG
options:
-h, --help show this help message and exit
--arm-dominance {left,right}
left/right arm wearing the Myo (default: right)
--data DATA path to the data directory to save recorded data (default: /Users/iomz/ghq/github.com/Interactions-HSG/MyoKTROS/data)
--duration DURATION seconds to record each gesture for recoding (default: 30)
--emg-mode EMG_MODE set the myo.types.EMGMode to calibrate with (1: filtered/rectified, 2: filtered/unrectified, 3: unfiltered/unrectified) (default: 1)
-g GESTURE, --gesture GESTURE
if specified, only record a specific gesture (default: all)
--knn-algorithm {auto,brute,ball_tree,kd_tree}
algorithm for fitting the knn model (default: auto)
--knn-k KNN_K k for fitting the knn model (default: 5)
--knn-metric {minkowski,euclidean,manhattan}
distance metric for fitting the knn model (default: minkowski)
--mac MAC specify the mac address for Myo (default: None)
--model-type {keras,knn,svm}
model type to calibrate (default: keras)
--n-samples N_SAMPLES
number of samples to detect a gesture (default: 50)
--svm-c SVM_C regularization parameter C for fitting the svm model (must be strictly positive) (default: 1.0)
--svm-degree SVM_DEGREE
degree of polynomial function for fitting the svm model with 'poly' kernel (default: 3)
--svm-gamma {scale,auto}
kernel coefficient for fitting the svm model (default: scale)
--svm-kernel {linear,poly,rbf,sigmoid}
kernel type for fitting the svm model (default: poly)
-v, --verbose sets the log level to debug (default: False)
--only-record only record EMG data without training a model (default: False)
--only-train only train a model without recording EMG data (default: False)
Connecting MyoKTROS to a robot
MyoKTROS currently supports the following robots.
UFACTORY
- xArm series: https://www.ufactory.cc/xarm-collaborative-robot/
- LITE 6: https://www.ufactory.cc/lite-6-collaborative-robot/
xarm_ros2
xarm_ros2 includes the ROS2 packages for xArm/Lite6 robot series.
The ROS nodes to communicate with xArm/Lite6 robots can be deployed by the included docker-compose.yml
.
MyoKTROS invokes the xarm_api services via internal WebSocket messages whose payloads are analogous to the SRV files for the corresponding services.
Note that, the ros2-xarm
container attempts to connect to the robot accessible at the IP address specified by the environmental variable ROBOT_IP
.
Depending on the model, you need to modify the launch command. For example:
- for Lite6 robots
ros2 launch xarm_api lite6_driver.launch.py robot_ip:="${ROBOT_IP}"
- for xArm7 robots
ros2 launch xarm_api xarm7_driver.launch.py robot_ip:="${ROBOT_IP}"
Finally, set myoktros.robot.driver
to xarm_rosws
and configure the parameters.
[myoktros.robot.xarm_rosws]
model = lite6 # the robot model
ip = 127.0.0.1 # the IP address for the ROS service client
port = 8765 # the port that the ROS service client's websocket interface is listening at
WebSocket API
UFACTORY's proprietary WebSocket API: https://github.com/xArm-Developer/ufactory_docs/blob/main/websocketapi/websocketapi.md
With this API driver, MyoKTROS can operate without ROS.
Set myoktros.robot.driver
to ufactory_ws
and configure the parameters.
[myoktros.robot.ufactory_ws]
model = lite6 # the robot model
ip = 10.0.0.6 # the IP address for the robot controller
Visualizing the State Machine
myoktros.Robot
is the base robot class for Robots to be intereacted with, and a default finite-state machine is implemented with transitions.
transitions provides two methods to draw the diagram for the state machines.
WebMachine
transitions-gui implements WebMachine
to produce a neat graph as a simple web service.
Run scripts/robot_web_machine.py
(startup may take a few momemnt) and access http://localhost:8080?details=true on your browser.
You may need additional dependencies if not with poetry:
pip install transitions-gui tornado
GraphMachine
The state machine diagram can also be drawn using Graphviz with the dot layout engine by scripts/robot_graph_machine.py
.
NOTE: pygraphviz cannot be installed straight for macOS, so not included in the poetry dependencies.
- Install Graphviz: see here
- Install python packages
- for macOS
pip install \ --global-option=build_ext \ --global-option="-I$(brew --prefix graphviz)/include/" \ --global-option="-L$(brew --prefix graphviz)/lib/" \ pygraphviz pip install graphviz
- Otherwise
pip install "transitions[diagrams]"
- for macOS
- Generate the diagram (gets saved in
assets/robot_state_diagram.png
)
./scripts/assets/generate_robot_state_diagram.py
Gestures
Any gestures can be registered in config.ini
, but these are some common examples.
REST
GRAB
STRETCH_FINGERS
EXTENSION
FLEXION
HORN
FLEMING
THUMBS_UP
SHOOT
TENNET
Myo
Myo Armbands are capable of streaming data as follows.
EMGData | FVData | IMUData | |
---|---|---|---|
Throughput | ~200 S/s | ~50 S/s | ~50 S/s |
Use scripts/speedometer.py
to see it yourself.
❯ ./scripts/speedometer.py -h
usage: speedometer.py [-h] [--emg-mode {0,1,2,3}] [--imu-mode {0,1,2,3}] [--mac MAC] [--seconds SECONDS]
Measure the data stream throughput from Myo
options:
-h, --help show this help message and exit
--emg-mode {0,1,2,3} set the myo.types.EMGMode (0: disabled, 1: filtered/rectified, 2: filtered/unrectified, 3: unfiltered/unrectified) (default: 1)
--imu-mode {0,1,2,3} set the myo.types.IMUMode (0: disabled, 1: data, 2: event, 3: all) (default: 0)
--mac MAC the mac address for Myo (default: None)
--seconds SECONDS the duration to record in seconds (default: 10)
Authors
- Iori Mizutani (@iomz)
- Felix Wohlgemuth
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file myoktros-0.3.0.tar.gz
.
File metadata
- Download URL: myoktros-0.3.0.tar.gz
- Upload date:
- Size: 20.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/4.0.1 CPython/3.11.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 221a799c44130601eb4cc6d5c5b3cc2c520a0f9b1c9821324fe418e595bc6e79 |
|
MD5 | 381a856166b435a3767104b02a149fea |
|
BLAKE2b-256 | fd6c0163e937aef6ad5bf17d72973626606205375ea772e53c528f6be1cfd6eb |
File details
Details for the file myoktros-0.3.0-py3-none-any.whl
.
File metadata
- Download URL: myoktros-0.3.0-py3-none-any.whl
- Upload date:
- Size: 18.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/4.0.1 CPython/3.11.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ce719e5997ae0f934a7da2967bda82e3e9c4ec6fe8d3774fadf70fed88df84b6 |
|
MD5 | adba01b65521c0e80792bb2ab7c336ad |
|
BLAKE2b-256 | 67bcf04285a7fc8d7aee4609486a34ec981679e1ea8c8b8b5d86b15f611ecea1 |