stac-keypoints-ui 0.0.1

3D GUI for mapping correspondences between 3D motion capture data and MuJoCo virtual biomechanical models.

stac-keypoints-ui

An user interface application that creates the correspondence between 3D motion capture landmarks and virtual animal sites.

Setup

NOTE: You must have Python version >=3.10 and pip installed on yourn machine, preferably in a package managing environment such as a distribution Anaconda/Miniconda on your local system to run the interface with ease.

Once you have fulfilled the aforementioned requirements, execute the command below to install.

pip install stac-keypoints-ui

To run the GUI, executed the command below.

stac_keypoints_ui mocap_data_file_path mujoco_xml_data_file_path

Supported Input Files

As you may have noticed from above, the program accepts two argument files. The first one is the 3D motion capture data and the second being the MuJoCo xml file for the virtual animal sites.

3D Motion Capture Data File

Three file types for motion capture data are supported: json, csv, and HDF5. The sections below will cover how to structure each file type.

HDF5 File Type

This file type should contain data in sort of a Python dictionary format. It should contain a key kp_data with its corresponding value being the motion capture data frames or timesteps each containing the x, y, z coordinates of each of the motion capture markers. Hence, this value should be an array with dimension or shape number of frames, number of keypoints/motion capture markers, 3. The last value of the dimension, 3, represents the [x_coordinates, y_coordinates, z_coordinates]. The second key-value pair should have a key kp_names with a value that is an array containing the names of the body parts corresponding to the motion capture markers. This value will have a shape or dimension of number of keypoints, meaning it is a one dimensional array with strings.

JSON

The json file should follow the same dictionary structure as that for the hdf5 file type which was specified above. So it should contain the kp_data and kp_names keys and their corresponding values as specfied above.

CSV

The csv format will be a little bit different from the rest. Take the example below:

frame_idx	mouth_x	mouth_y	mouth_z
0	3.6698393237494145	6.28660031286055	-1.5501244155883
1	4.13125883621487	-6.88562708648629	-2.06199066634826
2	4.61529499507544	-7.36953798404579	-2.48701100899353

The csv file should contain columns that contain a frame_idx that contains all the indices making up the rows of the csv file and each row will contain data on the x, y, z coordinates of each of the markers. However, in this case, the x, y, z coordinates will be separated and listed down as individual values as you can see in the table above. Hence, the following columns after frame_idx should be the individual x, y, z coordinates for each motion capture marker and the rows will make up the value for each frame. Please order them as body_part_1_x, body_part_1_y, body_part_1_z, body_part_2_x, body_part_2_y, body_part_2_z. Don't mix them up as this may cause issues with indexing. As a recap, the csv file should be structure like this:

frame_idx	marker_1_x	marker_1_y	marker_1_z	marker_2_x	marker_2_y	marker_2_z
0	x coordinate at frame 1	y coordinate at frame 1	z coordinate at frame 1	x coordinate at frame 1	y coordinate at frame 1	z coordinate at frame 1
1	x coordinate at frame 2	y coordinate at frame 2	z coordinate at frame 2	x coordinate at frame 2	y coordinate at frame 2	z coordinate at frame 2
2	x coordinate at frame 3	y coordinate at frame 3	z coordinate at frame 3	x coordinate at frame 3	y coordinate at frame 3	z coordinate at frame 3

You may have columns such as marker_1_reprojection_error or marker_1_n_cam. These values can be kept but will not be used. You will be fine as long as you do not mix around the columns of the markers. It is important to put down the x, y, and z columns for a marker before moving onto the next one.

MuJoCo XML File

This is simply the .xml file that is used for uploading virtual body models into the MuJoCo physics simulator program. When passed as a second file argument, the absolute coordinates of the model's geoms will be parsed from the file and used for plotting and generating correspondences. In the backend, the corresponding body element will actually be mapped to the keypoints to form a correspondence. The reason we used geoms for plotting and forming the initial correspondences is because geoms are less sparse than body elements since many geoms fall under body elements which are in the top-level in the hierarchy of mujoco xml elements.