Summary
Theia3D uses a large number of precisely predicted keypoints and an inverse kinematic approach to obtain whole body pose estimates during any movement - but how much is the solution dependent on the IK model and its constraints? In this blog post, we investigate this question by comparing kinematics produced with varying degrees of freedom in the lower limb joints, and find that the kinematic results are largely unchanged - a nod to the robustness of our model and its keypoints.
For the release of Theia3D Apollo, we decided to take our updated validations efforts one step further and perform a series of comparisons to investigate the effect of varying lower limb degrees of freedom (DOF) on our markerless tracking. As you may know, Theia3D has always used an inverse kinematic (IK) approach to pose estimation, which is a well-known approach to reach a globally-optimized pose solution based on landmarks or keypoints across multiple body segments. This method was widely used in the field of biomechanics long before Theia3D came along, and it’s a great approach that allows a linked set of rigid segments to be modeled with specified joint constraints. In its infancy, Theia3D had one IK model with no possible adjustments: 3 DOF at the ankle, 2 DOF at the knee, and 3 DOF at the hip. Since then, our model and software have come a long way, and in recent years additional models and joint constraint options have been introduced.
With the upcoming release of Theia3D Apollo, the available models have been modified slightly to offer simplified but improved options. Theia3D Apollo will include three model options as described briefly below:
Default Model
Consists of two kinematic chains: lower body (pelvis and legs) and upper body (torso, arms, and head).
No abdomen or neck segments.
Shoulder joints and head segment are allowed 6 DOF.
Ankle joints are allowed 6 DOF, with limited translation.
Separate Arm and Head Model
Consists of five kinematic chains: lower body (pelvis and legs), torso, left arm, right arm, and head. Additional chains allow tracking of limbs or body parts when some limbs cannot be tracked.
No abdomen or neck segments.
Shoulder joints and head segment are allowed 6 DOF.
Ankle joints are allowed 6 DOF, with limited translation.
Full Body Model
Consists of one, whole-body kinematic chain.
Abdomen and neck segments included.
Shoulder joints are allowed 6 DOF.
Head is allowed 3 DOF.
Ankle joints are allowed 6 DOF, with limited translation.
Across all of these models you may notice that the ankle and shoulder joints are allowed 6 DOF, and the knees can be selected to have 2 or 3 DOF. This represents a slight change from the current version, in which 6 DOF shoulder and ankle joints are a user-selected option (Use Free Arms and Use Free Feet). As our model and algorithms continue to improve and demonstrate increasing robustness, we are able to reduce constraints on the model without losing out on the accuracy or reliability of its tracking. This allows us to implement these specified joint constraints as the default, simplifying the modeling selection process for you, the user.
These types of changes don’t come about without significant testing though, which brings us back to the topic of the current blog; investigating the effect of varying degrees of freedom at the lower limb joints. For these tests, we looked at allowing 3 or 6 DOF at each of the ankle, knee, and hip joints. This means that besides being able to rotate fully independently, each lower limb joint would also allow 3D translations. Our ability to even include these degrees of freedom speaks to the number and stability of our model’s predicted keypoints, which provide more than enough reliably tracked points on each segment to determine their position and orientation, independent of all other segments. (To be mathematically determinate, three or more non-collinear points on each segment are required.) This may seem like a simple requirement, but it’s important to remember that even some marker-based models don’t make use of enough markers to have this level of segment pose determination.
To test these modeling options, we processed our concurrent validation dataset using multiple combinations of joint constraints for the ankle, knee, and hip, shown in the table below.
Setup | Ankle DOF | Knee DOF | Hip DOF |
1 | 3 | 3 | 3 |
2 | 6 | 3 | 3 |
3 | 3 | 6 | 3 |
4 | 3 | 3 | 6 |
5 | 6 | 6 | 6 |
Table 1: This table shows the combinations of varying joint DOFs we compared. Setup 1, using 3 DOF at all three of the lower limb joints has been used for most of our previous validation testing.
Using these combinations of joint constraints, we were able to look at the effect of the added joint mobility at each of the lower limb joints individually (Setups 2-4), and at the overall effect of allowing this mobility at all of the joints at once (Setup 5). As you might expect, we have very high expectations for the performance of our keypoint estimations, and therefore expected fairly consistent results between all of these models. And we’re glad to report that the results stand up to our high expectations!
Of particular interest for these comparisons were the lower limb segment angles and lower limb joint angles during our most standard evaluation movement, walking. In both cases, we found only minimal changes in the segment orientations and joint angles across the varying joint constraint configurations. These changes are visually summarized in the videos below.
When we compared the segment angles across the different configurations, we found that the only meaningful change that occurred with the changing joint constraints was a decrease in pelvis lateral tilt (Y-angle) range of motion across the gait cycle, when additional degrees of freedom were included. This change increased as the added degrees of freedom moved from distal to proximal joints (ankle to knee to hip), or when all joints were set to 6 DOF. This was accompanied by a very minor shift in the pelvis transverse plane rotation (Z-angle), and even smaller changes to the thigh and shank axial rotations (Z-angle) during early swing (60-80% gait cycle).
Figure 1: This video shows the lower limb segment angles compared to our reference marker dataset during walking, for each of the joint constraint configurations. Within this video, the results cycle through the different setup options in Table 1 above, preceding from setup 1 through 5 and starting again with setup 1. The joint constraints can be seen in the legend, where H, K, A represent the hip, knee, and ankle, respectively, and 3 or 6 represent the degrees of freedom at that joint.
Figure 2: This static figure shows the segment angles for the model with 3 DOF at all three joints against those for the model with 6 DOF at all three joints.
When we compared the joint angles across the varying DOF configurations, we primarily found decreased hip ab/adduction, which is evidently associated with the changes in the pelvis orientation that were previously noted. Other changes were very minor; slight changes in the frontal and transverse plane hip and knee joint angles, mostly during early swing (60-80% gait cycle).
Figure 3: This video shows the lower limb joint angles compared to our reference marker dataset during walking, for each of the joint constraint configurations. Within this video, the results cycle through the different setup options in Table 1 above, preceding from setup 1 through 5 and starting again with setup 1. The joint constraints can be seen in the legend, where H, K, A represent the hip, knee, and ankle, respectively, and 3 or 6 represent the degrees of freedom at that joint.
Figure 4: This static figure shows the joint angles for the model with 3 DOF at all three joints against those for the model with 6 DOF at all three joints.
Together, these changes are unsurprising - when we allow translation between the lower limb segments, positional influences of one segment within the kinematic chain on the other segments are removed. Here, we see this freedom manifest in the pelvis no longer being ‘pushed’ or ‘pulled’ vertically by each of the legs throughout the movement, primarily reducing the pelvis lateral tilt range of motion and other associated effects such as reduced hip ab/adduction.
We also looked at the results of this comparison for different movements including running, countermovement jumps, drop jumps, and squats. Across all of these movements we saw similarly minimal changes with changes to the model DOFs. For running, the changes were almost identical to those shown here for walking, with reduced lateral pelvis tilt and hip ab/adduction. For the non-cyclic movements, the changes were even less pronounced - the biggest difference was in the foot segment rotation in the sagittal plane and the associated ankle dorsi/plantarflexion, where the model with 6 DOF at the ankle had very slightly reduced peak angles.
However, one of the most notable findings from this investigation is actually just how limited an effect the changes in model joint constraints have on the kinematics; here we see that whether or not the lower limbs are solved using a constrained global optimization approach or are modeled with 6 DOF, the segmental orientations are mostly unchanged. This truly speaks to the robustness of our model’s predictive keypoints, which are accurate and stable enough to produce results that are not impacted by the method used to solve the whole body pose. Not only does Theia3D produce accurate, reliable keypoint predictions for easy-to-guess points like joint centers, but also for many more minor keypoints on each segment, allowing robust tracking of all segments regardless of model joint constraints. This is the kind of data-driven, biomechanical accuracy that we strive to deliver with our markerless motion capture software, and we think the results speak for themselves.
Check out our previous posts on Theia3D Apollo’s Updated Validation efforts and Model Definitions, and keep an eye on our blog in the upcoming weeks for additional content related to the approaching release. To learn more about Theia, click here to book a demo.
Comments