tactigon-gear 4.1.7

Tactigon Gear to connect to Tactigon Skin wereable platform

Tactigon Gear

This package enables the wearable device Tactigon Skin to connect to your python project using Bluetooth Low Energy.

SDK version: 4.1.3

Architecture

Tactigon Gear environment has the following architecture:

Server is located on the cloud and it is manteined by Next Industries s.r.l. Server has a web interface where you can handle your profile and your data (models and gestures)

Provided Tactigon Gear SDK is the implementation of Tactigon Gear environment client side

Tactigon Gear SDK is used for collecting new raw data, send the data to server, ask server to train a model using the raw data, and download the model from server. Finally use the model for testing real-time gesture recognition.

Prerequisites

In order to use the Tactigon Gear SDK the following prerequisites needs to be observed:

• Python version: following versions has been used and tested. It is STRONGLY recommended to use these ones depending on platform.

  • Win10: 3.8.7
  • Linux: 3.8.5
  • Mac osx: 3.8.8
  • Raspberry: 3.7.3

• It is recommended to create a dedicated python virtual environment and install the packages into the virtual environment:

  • python -m venv venv
  • pip install tgear-sdk

• Depending on your installation (Linux, Raspberry, Mac users) you may need to use python3 and pip3 instead of python and pip respectively

Licensing

In order to perform new training and download them you need to register on following web side: https://www.thetactigon.com/ai/web/ Once registration is done you can go to Profile section and click on Json File button to download file user_data.json The use of this file is described later in this doc.

Tactigon Skin

Status led

Tactigon Skin led indicates current fucntional state:

• when not paired to any device it rotates red/green/blue led
• when paired (connected to PC/raspberry/smartphone) but not transmitting any data, blue led blinks
• when paired and transmitting data, grean led blinks

Tactigon Gear examples

For a list of example you can go to Tactigon SDK repository

Tactigon Gear CLI instructions

To run the Tactigon Gear CLI open a command window (cmd/powerShell in Windows, terminal windows for all the others O.S.), move to installation folder and run the python script cli.py with the following command:

• python cli.py

The following menu will appear:

Gear SDK Client, select an option:
Enter 0 to test server connection
Enter 1 to data collection
Enter 2 to send data to server
Enter 3 to train a new model
Enter 4 to download a model from server
Enter 5 to simple gesture recognition
Type anything to EXIT

Regardless of choosen menu item, system is ready to use when showing message reportind succesfully connection to Tactigon Skin. Succesfully connection is also confirmed by Tactigon Skin led (see next paragraph)

Before start using the SDK user needs to change the config files according to his needs.

Linux/raspberry users: if working in SSH shell you may need to specify a DISPLAY with something like this:

• export DISPLAY=:0.0

Tactigon Gear CLI configuration files

Before using the SDK some general information must be set in the two following config files:

• user_data.json : Locally store login information. Need to match credentials on the server side

  {
   "user_name": "joe",
   "auth_key": "_abcde123456__"
  }
  

• hal.json : This config file store all necessary hardware related information

  {
  "BLE_RIGHT_ADDRESS": "BE:A5:7F:47:54:65", # Tactigon Right BLE Address (See Find Tactigon BLE Address section)
  "BLE_LEFT_ADDRESS": "BE:A5:7F:47:54:65",  # Tactigon Left BLE Address (See Find Tactigon BLE Address section)
  "BLE_RIGHT_ENABLE" : "True",  # Tactigon Right enable flag
  "BLE_LEFT_ENABLE" : "True",   # Tactigon Left enable flag
  "SERIAL_COM_PORT": "COM3",  # No need to change - debug 
  "NUM_SAMPLE": 10,           # No need to change - debug
  "INTERFACE" : "Bluetooth"   # No need to change - debug
  }
  

• client.json : This config file store http client related information

  {
  "SERVER_URL": "https://tgear.eu.pythonanywhere.com", # Tactigon Gear Server IP Address
  "TRAINING_SESSIONS": ["26_02_2021_1", "26_02_2021_2"], # list of session to upload (see Training section)
  "MODEL_SESSIONS": ["26_02_2021_1", "26_02_2021_2"], # list of session for model building (see Model Building section)
  "MODEL_GESTURES": ["circle", "square"], # list of gestures recognized by the model (see Model Building section)
  "MODEL_NAME": "MODEL_RIGHT", # name assigned to the requested model (see Model Building section)
  "MODEL_DATA_PATH": "data/models/",  # No need to change
  "USER_DATA_PATH": "config_files/user_data.json",  # No need to change
  "RES": {                    # No need to change - debug 
    "user": " ",              # No need to change - debug 
    "password": " ",          # No need to change - debug  
    "status": "unsuccessful"  # No need to change - debug 
  }
  

Tactigon Gear CLI training session

• Before running a training session the /config_files/data_collection.json data must be edited:

  • data_collection.json : This config file store all necessary information for data collection

  {
  "RAW_DATA_PATH": "./data/raw/",     # path where you want to store raw data - no need to change
  "GESTURE_NAME": ["square_cw","circle_acw"],   # list of gestures you want to record
  "SESSION_INFO": "19_02_2021_A",     # recording session information
  "HAND": "RIGHT"                     # which hand is going to be used (right/left)
   }
  

• Turn ON the Tactigon used for the training session

• Run the main.py python script

• Select option 1 Enter 1 for data collection

• Wait for te Tactigon to be connected and activate gesture recognition by pressing button 4 on the Tactigon

• Follow console instructions to train all the gestures entered in the data_collection.json config file

• At the end of the training select option 8 Enter 9 for EXIT to exit the client

Tactigon Gear CLI Training Data Upload

Once user has run one or more training session (even with different gesture sets), training data can be uploaded to the server and used later on to generate a Model. To do that follow the instructions below:

• Edit the "TRAINING_SESSIONS" field int theclient.json config file including all the sessions that user wants to upload to the server. Session are indicated by the "SESSION_INFO" field in the data_collection.json config file used during training.
• Run the main.py python script
• Check server connection select option 0 Enter 0 to test server connection
• Select option 2 Enter 2 to send data to server
• Enter 9 for EXIT to exit the client

Tactigon Gear CLI Model Building Instructions

• Edit the "MODEL_SESSIONS" field int theclient.json config file including all the sessions that user wants to use for building the model. Session are indicated by the "SESSION_INFO" field in the data_collection.json config file used during training.
• Edit the "MODEL_GESTURES" field int theclient.json config file including all the gestures that the model has to recognize.
• Edit the "MODEL_NAME" field int theclient.json config file to assign a name to the generated model.
• Run the main.py python script
• Check server connection select option 0 Enter 0 to test server connection
• Select option 3 Enter 3 to train a new model
• A new model is created on server side and stored to DataBase
• Enter for EXIT to exit the client

Tactigon Gear CLI Model Download Instructions

• Edit the "MODEL_NAME" field int theclient.json config file to select the model to download from server.
• Select option 4 Enter 4 to download model from server

Tactigon Gear CLI Gesture Recognition Setup and Test

• Before using the gesture recognition feature the /config_files/real_time.json data must be edited:

  • real_time.json : This config file store the models name used by the gesture recognition engine

  {
  "MODEL_NAME_RIGHT": "MODEL_RIGHT",  # name of the model used for right hand gesture recognition
  "MODEL_NAME_LEFT": "MODEL_LEFT",    # name of the model used for left hand gesture recognition
  "MODEL_PATH": "data/models/" # No need to change
   }
  

• Run the cli.py python script

• Select option 5 Enter 5 for simple gesture recognition

• Recognized gestures will be printed on the terminal

• Enter 9 for EXIT to exit the client