Metadata-Version: 2.4
Name: stac_keypoints_ui
Version: 0.0.1
Summary: 3D GUI for mapping correspondences between 3D motion capture data and MuJoCo virtual biomechanical models.
Author-email: Rusham Bhatt <rbhatt4@umbc.edu>, Scott Yang <yuy004@ucsd.edu>, Talmo Pereira <talmo@salk.edu>
License-Expression: MIT
Project-URL: Homepage, https://github.com/talmolab/stac-keypoints-ui
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: h5py>=3.0
Requires-Dist: mujoco
Requires-Dist: numpy
Requires-Dist: pygfx
Requires-Dist: pyside6
Requires-Dist: wgpu
Requires-Dist: glfw
Requires-Dist: pandas
Requires-Dist: pyyaml
Requires-Dist: pip
Requires-Dist: thin-plate-spline>=1.2.0
Dynamic: license-file

# stac-keypoints-ui
An user interface application that creates the correspondence between 3D motion capture landmarks and virtual animal sites.

## Setup
*NOTE:* You must have Python version >=3.10 and pip installed on yourn machine, preferably in a package managing environment such as a distribution Anaconda/Miniconda on your local system to run the interface with ease.

Once you have fulfilled the aforementioned requirements, execute the command below to install.
```bash
pip install stac-keypoints-ui
```
To run the GUI, executed the command below.
```bash
stac_keypoints_ui mocap_data_file_path mujoco_xml_data_file_path
```

## Supported Input Files
As you may have noticed from above, the program accepts two argument files. The first one is the 3D motion capture data and the second being the MuJoCo xml file for the virtual animal sites.

### 3D Motion Capture Data File
Three file types for motion capture data are supported: json, csv, and HDF5. The sections below will cover how to structure each file type.

#### HDF5 File Type
This file type should contain data in sort of a Python dictionary format. It should contain a key `kp_data` with its corresponding value being the motion capture data frames or timesteps each containing the x, y, z coordinates of each of the motion capture markers. Hence, this value should be an array with dimension or shape `number of frames, number of keypoints/motion capture markers, 3`. The last value of the dimension, 3, represents the `[x_coordinates, y_coordinates, z_coordinates]`. The second key-value pair should have a key `kp_names` with a value that is an array containing the names of the body parts corresponding to the motion capture markers. This value will have a shape or dimension of `number of keypoints, ` meaning it is a one dimensional array with strings.

#### JSON
The json file should follow the same dictionary structure as that for the hdf5 file type which was specified above. So it should contain the `kp_data` and `kp_names` keys and their corresponding values as specfied above.

#### CSV
The csv format will be a little bit different from the rest. Take the example below:
| frame_idx | mouth_x | mouth_y | mouth_z |
|---|:--:|:--:|:--:|
|0|3.6698393237494145|6.28660031286055|-1.5501244155883|
|1|4.13125883621487|-6.88562708648629|-2.06199066634826|
|2|4.61529499507544|-7.36953798404579|-2.48701100899353|

The csv file should contain columns that contain a `frame_idx` that contains all the indices making up the rows of the csv file and each row will contain data on the x, y, z coordinates of each of the markers. However, in this case, the x, y, z coordinates will be separated and listed down as individual values as you can see in the table above. Hence, the following columns after `frame_idx` should be the individual x, y, z coordinates for each motion capture marker and the rows will make up the value for each frame. Please order them as `body_part_1_x, body_part_1_y, body_part_1_z, body_part_2_x, body_part_2_y, body_part_2_z`. Don't mix them up as this may cause issues with indexing. As a recap, the csv file should be structure like this: 
| frame_idx | marker_1_x | marker_1_y | marker_1_z | marker_2_x | marker_2_y | marker_2_z
|---|:--:|:--:|:--:|:--:|:--:|:--:|
|0|x coordinate at frame 1|y coordinate at frame 1|z coordinate at frame 1|x coordinate at frame 1|y coordinate at frame 1|z coordinate at frame 1|
|1|x coordinate at frame 2|y coordinate at frame 2|z coordinate at frame 2|x coordinate at frame 2|y coordinate at frame 2|z coordinate at frame 2|
|2|x coordinate at frame 3|y coordinate at frame 3|z coordinate at frame 3|x coordinate at frame 3|y coordinate at frame 3|z coordinate at frame 3|

You may have columns such as `marker_1_reprojection_error` or `marker_1_n_cam`. These values can be kept but will not be used. You will be fine as long as you do not mix around the columns of the markers. It is important to put down the x, y, and z columns for a marker before moving onto the next one.

### MuJoCo XML File
This is simply the .xml file that is used for uploading virtual body models into the MuJoCo physics simulator program. When passed as a second file argument, the absolute coordinates of the model's geoms will be parsed from the file and used for plotting and generating correspondences. In the backend, the corresponding body element will actually be mapped to the keypoints to form a correspondence. The reason we used geoms for plotting and forming the initial correspondences is because geoms are less sparse than body elements since many geoms fall under body elements which are in the top-level in the hierarchy of mujoco xml elements.
