Summary: In this blog post, we provide some guidance for working with Theia3D data in Visual3D.
When considering the functionality of Theia3D, there are several obvious benefits that immediately come to mind: no markers, which reduces the time and effort to collect data and the burden on subjects; no environmental requirements, which permits more realistic out-of-lab data collections; and no marker tracking, which eliminates a tedious manual step from the data processing pipeline. However, one often overlooked benefit is in the post-processing and analysis stage, where Theia Markerless continues to simplify workflows and provide serious time-savings through its integration with Visual3D.
Since the introduction of Theia3D, Visual3D has been refined with a focus on markerless outputs to ensure seamless integration and to continue to offer the most comprehensive tools for analyzing human movement. Now, even the post-Theia3D processing steps in your data analysis procedures are simplified and expedited through the use of Visual3D. The purpose of this blog post is to describe the general workflow and some technical aspects of working with markerless data in Visual3D.
Theia .C3D Outputs
The .C3D file format has long been the standard for biomechanics data, and so it is only natural that Theia3D exports .C3D files as a primary data output. To obtain .C3D files using Theia software, you can either use Theia3D to process an individual trial or TMBatch to batch process multiple trials at once.
When working with an individual trial within Theia3D, use the Save Skeleton Poses button to save .C3D files for any tracked people once the trial has been fully processed. This allows you to select which tracked person(s) should be included in the exported file and the frame range for the exported file.
When processing several trials through TMBatch, ensure the Save C3D option is selected to save .C3D files for any tracked person(s) for all batch-processed trials. This option will save each tracked person’s movement in separate files.
It is important to note that regardless of how you save your .C3D files (Theia3D vs TMBatch), you will always have access to two versions of the tracked movement: the unfiltered version (e.g. pose_0.c3d) and the filtered version (e.g. pose_filt_0.c3d). When processing a movement trial, Theia3D will apply the built-in filter to smooth the tracking and reduce any noise. In general, it is recommended to set the filter cutoff frequency specifically for the movement in your trial and to use the filtered .C3D file, as this allows the model joint constraints to be maintained (which would not be the case when applying post-processing filtering to the unfiltered .C3D). If the GCVSPL Cutoff Frequency is not changed, the default value of 20 Hz will be used.
Once you have the .C3D files for your movement trial(s), you are ready to begin working with the data in Visual3D. Theia-generated .C3D files can be opened in Visual3D by double-clicking the file, or using any ‘File Open’ options within Visual3D, such as via the Command Pipeline or the Open button.
Figure 1: New Visual3D workspace created by opening a Theia-generated .C3D file.
Upon opening the .C3D file, you should be greeted with the following view showing your opened file (under Motion Files) with an automatically-assigned model (under Model/Calibration Files).
Figure 2: Example of the Model/Calibration Files and Motion Files columns in a Visual3D workspace.
If you are unsure of any settings that were selected when the trial was processed by Theia3D, you can review the PARAMETERS folder of the data tree, within the Signals and Events tab. This will provide you with information about the version of Theia software that was used to produce the file as well as some of the adjustable settings used in Theia, such as Max People, 3DOF Knee, Free Arms, Free Feet, and the GCVSPL Cutoff Frequency.
Figure 3: Theia3D Parameter signals available in Visual3D, which provide information about how the movement trial was processed.
Visual3D Model
One of the ways in which Theia Markerless .C3D files integrate cleanly into Visual3D is through the application of an automatic model template in Visual3D. This means that there is no need to define or construct your own biomechanical model based on landmarks in a static trial, which is often a very time-consuming process.
Besides removing the need for you to decide on and implement a specific biomechanical model based on marker positions, the use of an automatic model ensures consistency across all individuals, studies, locations, and users. This inherent consistency simplifies efforts to pool data from different sites, ensuring that there are no issues with marker placement, marker sets, or any of the other potential issues that must be addressed when using markers.
And even though an identical model template is applied to all individuals’ .C3D files, the model itself is still subject-specific, utilizing segment lengths and other scaling factors from the model created for the person by Theia3D and conveyed through the .C3D file. However, be aware that the Mass and Height metrics within the Subject Data / Metrics tab are default values within Visual3D, not subject-specific values from Theia.
Figure 4: The Visual3D Models tab, illustrating the automatic model applied to Theia-generated .C3D files.
All this being said, there are a few specific aspects of the automatic model applied in Visual3D that are important to be aware of:
First, the model includes two Thorax or Thorax/Ab segments, both of which track the same signals throughout the movement trial, but their definitions differ slightly. The Thorax segment is a kinematic-only segment, defined such that its origin is at the base of the neck, or the proximal end of the thorax segment. The Thorax/Ab segment is a kinetic segment, and is defined such that its origin is contiguous with the origin of the Pelvis segment, at the distal end of the thorax segment. Besides the slight difference in segment definition, either of these segments could reasonably be used for the calculation of segment or joint angles such as the pelvic-thorax angle, or as the reference segment for the shoulder joint angles.
Second, the Visual3D model is a 6DOF model, meaning that there are no joint constraints enforced within Visual3D and each segment is tracked with six degrees of freedom. However, this modeling decision is to allow the joint constraints enforced by the IK model in Theia to be the only constraints; these are maintained through the segment pose information conveyed within the .C3D files, which include the information to model each segment with six degrees of freedom.
Third, there are no built-in angle offsets that are used to normalize joint angle calculations. Some users who construct their own models in Visual3D would be familiar with establishing a baseline joint angle exhibited by the participant during the static pose. This value can then be subtracted from the joint angle values calculated during movement trials to obtain signals that are relative to the individual’s baseline static posture. However, since the Visual3D model does not utilize a static posture in its definition, all joint angles are calculated as the full angle values between the adjacent segments, with no baseline values subtracted.
Finally, although the Visual3D model is applied automatically, it is possible (although not necessary) to modify the model or use your own model template in place of the default model. If you have a particular need to modify the model, you will also need to make sure that ‘Ignore calibration’ is deselected for any modified segments, which will ensure that the segment definition used in the Models tab is used for the movement trials too.
Figure 5: The Tracking Targets portion of a segment definition tab within Visual3D, highlighting the Ignore calibration option which should be disabled for any modified segments.
ROTATION Data Type
When it comes to conveying the person’s pose throughout the movement trial in Theia .C3D files, new users may wonder how exactly this is achieved. Are there TARGET or LANDMARK signal types which contain signals for the key points identified by Theia? Does the output include specific anatomical landmarks that I can use? This is another aspect that differs from what motion capture users may be used to.
Theia .C3D files use the ROTATION signal type to convey the person’s movement and drive the Visual3D model. This signal type is described in greater detail in the Visual3D documentation, but briefly it includes the six degree-of-freedom pose (position and orientation) information for each individual segment’s local coordinate system. This information is contained in 4x4 transformation matrices, which are displayed as 16 column components for each 4x4 matrix signal, at each frame of data throughout the movement. The 16 column components (numbered 0-15) can be interpreted as follows to form the 4x4 matrix at each frame:
0 | 1 | 2 | 3 | R11 | R12 | R13 | X | |
4 | 5 | 6 | 7 | where | R21 | R22 | R23 | Y |
8 | 9 | 10 | 11 | R31 | R32 | R33 | Z | |
12 | 13 | 14 | 15 | 0 | 0 | 0 | 1 |
These signals aren’t typically of much use themselves, except possibly the translation elements [3, 7, 11] or [X, Y, Z] which provide the position of the local coordinate system’s origin in global space. Each segments’ local coordinate system has its origin at the proximal joint of the segment, so these values can be used to obtain 3D joint positions. For example, using the signals [3, 7, 11] from the l_foot_4X4 signal will provide the 3D position of the left ankle joint. This is just one of many ways the joint center position can be obtained in Visual3D.
Automatic Signals
With the application of the automatic Visual3D model comes a selection of signals associated with that model that can be useful. The primary signal types for these signals are LANDMARK and KINETIC_KINEMATIC.
The automatic Visual3D model includes LANDMARK signals for only two specific anatomical positions, which are the left and right heel landmarks (L_HEEL, R_HEEL). These signals can be very useful for event detection, particularly kinematic gait events which often use heel positions to generate heel-strike events.
The automatic model also generates numerous KINETIC_KINEMATIC signals for each segment of the model. These are contained in separate folders for each segment, the short-hand names of which are listed here. This includes various signals such as the segment angular acceleration, angular velocity, center of gravity acceleration, center of gravity velocity, center of gravity position, and so on. In many cases, these automatically-generated signals are already a great starting point for obtaining useful biomechanical signals from your Theia .C3D files.
Figure 6: Example signals generated by the automatic Visual3D model.
Visual3D Pipelines
For all other processing tasks usually undertaken in Visual3D such as event generation, link-model-based calculations, and signal exporting, these can be easily accomplished using Visual3D’s powerful and flexible Pipeline Workshop. This tool is chock full of commands that can allow you to efficiently process your data in a consistent manner across workspaces. In all likelihood, any existing processing pipelines you have developed for marker-based processing can be quickly and easily adapted to work with markerless data, if any adaptation is needed at all. Besides the removal of any model-building steps, most commands and calculations would only need to be checked for dependencies on specific TARGET or LANDMARK signals associated with your marker set or custom model template, and an adequate alternative signal associated with the markerless model should be selected. Link-model-based calculations such as joint angles, metrics such as peak values or signal values at events, and most other signal handling steps can likely remain the same or operate with minor adjustments, enabling a quick and easy transition from using such pipelines with marker-based to markerless data.
Conclusion
Overall, the integration of Theia data in Visual3D allows these two programs to operate almost seamlessly in taking your data from raw videos to complex biomechanical signals, ready for statistical analysis, reporting, or publishing in only a few simple steps. Say goodbye to the days of manual modeling and messy marker data, and welcome to streamlined markerless data analysis.
To learn more about Theia, click here to book a demo.