Summary: This is the first of our Theia release blogs and the message is simple: "Things were good, and now they are better". We see subtle, but meaningful improvements in challenging poses (3D position and orientation of the multibody model) with large ranges of motion, and we see reduced variability for many signals. We are committed to documenting and “re-validating” each release as standard practice, with a goal to continually make each release document more comprehensive.
While we would like to rely on the scientific literature for independent assessments of our technology, the nature of print publications makes revisions of articles challenging and, therefore, there are limited updates on the performance improvements to Theia. While software improvements should trigger the amendment of articles, this is almost impossible with print publications. While archived pre-prints are considerably more flexible in accommodating documented revisions, this process is time consuming and for the researcher who published the article, what is the benefit? In a lot of cases, that same researcher is on another project, or grant, and revisiting analyses as software changes is not really practical for them. Furthermore, I can’t stress enough that our approach to software development is via data and validation, so providing existing customers and prospects with the most accurate and up to date information on the capabilities of the software is critical.
In our case, there are a variety of factors that can influence the accuracy of the results. These include, but are not limited to: adding new salient features, increasing the width and depth of the data set, and improving algorithm performance, all of which can influence the biomechanical output signals. Internally, we run a set of tests to assess how our development decisions are influencing results, but until now, we did not have a good way to communicate this information to users. This is the first of many blog posts that we will publish in an effort to close this gap. These posts will document and demonstrate the effects of our changes on the validation data that was used as the accuracy baseline for the software, for each major release.
This blog is long, so sorry! I was thinking about making this into a five part blog, and I’m sure marketing would have been happy with that, but oh well; please be patient while reviewing the figures and the assessment. So yes, this means that we have made a release available called Theia 2023.
In this post, we are going to cover the following movements: walking, running, double leg drop jump, single left drop jump, countermovement jump, and finally, squatting. The goal is to provide a comprehensive comparison between the new version of Theia, the previous version, as well as the marker based data. The sample population, the experimental protocol and analysis methods for some of these movements can be found in the reference papers listed at the end of this article.
Overall accuracy summary: Things are going in the right direction. We saw incremental and promising improvements that we believe demonstrate increased accuracy of lower limb tracking during all studied movements. Importantly, for these movements, these differences are within the error associated with marker placement and soft tissue artifact.
Walking:
As the movement that is likely studied the most during motion capture validation efforts, walking has always held an important role in our comparisons, against previous versions of Theia, and marker-based motion capture data. For this release, we saw the greatest improvements in hip joint center estimation, global pelvis pose, and ankle ab/adduction (inv/eversion).
The hip joint centers are now considerably closer to each other, more anterior, and more proximal relative to their previous positions, reducing the overall 3D position differences relative to the marker-based data by nearly 0.4 cm. The hip widths (joint center to joint center) measured by this version now have an identical mean to that measured by the marker-based system, with reduced variability. This is consistent with the reality that the inter-femoral head distance varies little across individuals, which has been reported to have standard deviations of 1 cm or lower (Hara et al., 2016; Mullaji et al., 2010)(Walking, Figure 1).
The global pelvis segment angles for this release demonstrated improved range of motion for the anterior/posterior tilt angle, and improved agreement in frontal plane pelvic tilt, relative to the marker-based reference data. The anterior/posterior tilt angle showed slightly more anterior tilt relative to the neutrally-defined marker-based pelvis, which leads to a small offset in the hip joint flexion/extension angles. Meanwhile, the hip joint ab/adduction angle was more similar between systems, thanks to the improvement in frontal plane pelvis tilt angle (Walking, Figure 2)(Walking, Figure 3).
Finally, the ankle ab/adduction (inversion/eversion) angles showed improved agreement with the marker-based data, thanks to improvements in the foot segment angles about the longitudinal axis. These improvements are observed despite a slight increase in the offset between the foot global segment z-angle, which captures toe-in/toe-out angles. This angle demonstrated excellent correlation with the the marker-based reference data across the whole gait cycle, but with a near-constant offset of about 6 degrees. (Walking, Figure 3) (Walking, Figure 4).
Hara, R., McGinley, J., Briggs, C., Baker, R., & Sangeux, M. (2016). Predicting the location of the hip joint centres, impact of age group and sex. Scientific reports, 6(1), 37707.
Mullaji, A., Shetty, G. M., Kanna, R., & Sharma, A. (2010). Variability in the range of inter-anterior superior iliac spine distance and its correlation with femoral head centre. A prospective computed tomography study of 200 adults. Skeletal radiology, 39, 363-368.
Walking, Figure 1: Hip widths between marker, Theia 2022, and Theia 2023. Theia 2023 showed similar means, and reduced variability to marker based measurements
Walking, Figure 2: Pelvis segment angles for walking (Marker based, Theia 2022 (teal), and Theia 2023 (blue))
Figure 3: Lower body joint angles for walking (Marker based, Theia 2022 (teal), and Theia 2023 (blue))
Walking, Figure 4: Foot segment angles during walking (Marker based, Theia 2022 (teal), and Theia 2023 (blue))
Running:
The running comparison shows more of the same as the walking (you will see this trend throughout all movements). Why running? Well, it’s faster, studied a lot, and people like to run! It also puts the systems through a larger range of motion.
We found that this release provides the following incremental improvements: reductions in hip joint center differences, improvements in swing-phase thigh and shank longitudinal axis angles, and improvements in foot segment longitudinal axis angles. The shank and thigh segment longitudinal axis angles lead to increased agreement in knee joint ab/adduction and internal/external rotation angles with the marker-based reference data, especially during the second half of the running gait cycle (Running, Figures 1 & 2).
Note: Since the collection of these data, we have modified recommendations in terms of camera placement to better capture the motion of the foot. These cameras were mounted fairly high on tripods and a truss system, and viewing angles of the foot were predominantly from the top. We have seen improvements in agreement between marker based and markerless for foot kinematics when the cameras are placed lower, and in line with our current recommendations.
Running, Figure 1: Joint angles during running (Marker based, Theia 2022 (teal), and Theia 2023 (blue))
Running, Figure 2: Segment angles during running (Marker based, Theia 2022 (teal), and Theia 2023 (blue))
Countermovement jump:
This movement was examined for a few reasons: it is non-cyclical, has a large range of motion, and has been used effectively to determine injury risk, so why not see how well we can track it! These results show excellent agreement between systems for all flexion/extension angles, and hip ab/adduction and internal/external rotation angles. Among the sagittal plane joint angles, the biggest observed difference (between Theia versions, and between Theia and marker-based) was a reduced plantarflexion angle during the jump flight phase. Among the other joint angles, similarities were observed in the mean joint angle patterns, especially at the hip joint, with reduced amplitudes captured by Theia 2023, with the exception of pelvis flexion, where more amplitude was measured, which is likely caused by skin artifact during high flexion for the marker based system. Most notably however, is the reduced variability observed in Theia frontal and transverse plane joint angles for all joints, except the hip ab/adduction angles, compared to the marker-based reference data. The high level of variability, relatively large magnitude, and similar patterns observed in these angles may indicate higher levels of kinematic crosstalk and marker movement which is not present in the Theia signals (Countermovement, Figures 1 & 2). It is also worth noting that the mean +/- standard deviation of the Theia 2023 joint angles remain within that for the marker-based data for all joint angles except ankle flexion during flight (as mentioned above) and ankle internal/external rotation immediately preceding and following the jump phase (20-30%, 60-70%).
Countermovement jump, Figure 1: Joint angles during a countermovement jump (Marker based, Theia 2022 (teal), and Theia 2023 (blue))
Countermovement jump, Figure 2: Segment angles during a countermovement jump (Marker based, Theia 2022 (teal), and Theia 2023 (blue))
Drop jump (double leg):
The comparison results for double-legged drop jumps are consistent with those for countermovement jumps, showing excellent agreement for sagittal plane joint angles and hip ab/adduction angles. Once again, the largest differences in sagittal plane joint angles were a reduced plantarflexion angle during the flight phase. Similar observations can be made about the frontal and transverse plane joint angles for the drop jump as were made for the countermovement jump (unsurprisingly). The marker-based reference data captured larger magnitudes and variability in these joint angles compared to those from Theia, in some cases showing large ranges of motion where the markerless joint angles are considerably smaller. Relative to previous versions of Theia, the agreement in mean knee ab/adduction angles and hip internal/external rotation angles has increased (Drop Jump (double leg), Figures 1 & 2). Once again, the mean +/- standard deviation for the Theia 2023 data remains within that for the marker-based data for all joint angles throughout the drop jump, except ankle flexion/extension during flight, and ankle ab/adduction and internal/external rotation immediately preceding and following the jump.
Drop jump (double leg), Figure 1: Joint angles during a drop jump (double leg)(Marker based, Theia 2022 (teal), and Theia 2023 (blue))
Drop jump (double leg), Figure 2: Segment angles during a drop jump (double leg)(Marker based, Theia 2022 (teal), and Theia 2023 (blue))
Drop jump (single leg):
The left single-legged drop jump kinematics demonstrated similar results as those for the double-legged drop jumps (and for the right single-legged drop jumps, not shown for brevity). In the sagittal plane, there was strong agreement between Theia 2023 and the reference marker-based data, with a slightly reduced plantarflexion angle and increased hip flexion angle during the flight phase. Hip ab/adduction angles also had high agreement and similar levels of variability, while the hip internal/external rotation angles captured similar mean patterns but different magnitudes and levels of variability. The knee ab/adduction joint angles from this release of Theia capture a similar movement pattern but opposite sign as those from the marker-based data. The frontal plane knee and ankle angles, and transverse plane angles at all joints have significantly reduced levels of variability compared to the marker-based data (left data shown, though results for the right side are effectively the same) (Drop jump (single leg): Figures 1 & 2).
Drop jump (single leg), Figure 1: Joint angles during a drop jump (single leg) (Marker based, Theia 2022 (teal), and Theia 2023 (blue); ipsilateral leg (solid), contralateral leg (dashed))
Drop jump (single leg), Figure 2: Segment angles during a drop jump (single leg) (Marker based, Theia 2022 (teal), and Theia 2023 (blue); ipsilateral leg (solid), contralateral leg (dashed))
Note to reader: In hindsight, it may have been a better idea not to show both the right and the left sides on the same plot… feedback taken. Consider it a work in progress.
Squats:
Last but not least, squats. The results from this release provide similar results in the sagittal plane joint angles to those from previous versions, but most notably providing improved agreement relative to the marker-based reference during the ‘down’ and ‘up’ portions of the squat motion. At the deepest section of the squat, the hip flexion angles diverge from the marker-based angles, however the marker-based signals may underestimate the real hip flexion due to marker occlusions and other difficulties in tracking the pelvis in such a deep squat posture. We also see strong agreement of the mean angles in hip ab/adduction and internal/external rotation, but with larger variability in the marker-based internal/external rotation angles. For the knee and ankle frontal and transverse plane joint angles, we observed the same general pattern in the mean angles as in the marker-based data, however with significantly reduced amplitudes and reduced variability (Squats, Figures 1 & 2).
Squats, Figure 1: Joint angles during a squat (Marker based, Theia 2022 (teal), and Theia 2023 (blue))
Squat, Figure 2: Segment angles during a squat (Marker based, Theia 2022 (teal), and Theia 2023 (blue))
If this type of comprehensive assessment of system performance is in line with your values, send us an email and we can tell you more.
References to original papers, please cite these and not this blog!!!!
Kinematics: https://doi.org/10.1016/j.jbiomech.2021.110665
Spatiotemporal parameters: https://doi.org/10.1016/j.jbiomech.2021.110414
Repeatability: https://doi.org/10.1016/j.jbiomech.2021.110422
A huge thank you to Rob Kanko for putting together most of this. Without his help in the validation work and on-going assessment, reporting on this information would not be possible!
To learn more about Theia, click here to book a demo.
Comments