Summary
In this blog post, we outline the newest features introduced through the upcoming Theia3D Apollo software release.
If you’ve been keeping up with our blog recently you’ll have seen the three posts we released last week providing updates on our validation results, model definitions, and degree of freedom testing on our model. While these posts are important reads in order to understand biomechanically-relevant changes to the software and our models, they don’t give you much to look forward to in terms of user-focused changes to the software, functionality, and available tools. So, here we finally give you a sneak preview of some of the features we’re most excited about, introduced for Theia3D Apollo.
Lens calibration visual feedback
For Theia3D users with camera systems requiring lens calibration, one of the most important but challenging steps in getting started with the new camera system is performing the intrinsic lens calibration. A strong intrinsic calibration is absolutely crucial to avoid possible downstream effects on the extrinsic calibration and movement tracking of people, and one of the crucial aspects of lens calibration is achieving a high coverage value for all camera views.
Previously, achieving high coverage for all camera views took attention to detail, careful performance of the calibration, and often, a few trial-and-error attempts to get it right. For Theia3D Apollo, we have introduced a new mode of visual feedback for lens calibrations, which is a green heatmap showing the area covered and the number of frames used to calibrate different areas within the camera view (via the intensity of green). This means you can easily identify specific areas within the camera views that were neglected, and approach your next attempt more informed. We hope this will allow new users to more easily achieve excellent intrinsic lens calibrations, and get started with data collections more quickly.
Figure 1: New heatmap visual feedback for lens calibrations, where the green area represents the calibrated portions of the camera views and the intensity represents the number of frames used to calibrate any given part of the image.
Chessboard calibration visual feedback
One of the most important aspects of obtaining high quality markerless motion capture data is ensuring that your camera system calibration is strong. For users who utilize the chessboard calibration method, this means taking care when recording the calibration, and committing some effort to hone your technique over time. One of the best ways to improve your technique is by reviewing the results of your calibrations after they have been processed, and taking note of what works and what doesn’t.
After processing a chessboard calibration, Theia3D provides feedback within the 2D camera views in the form of a zig-zag indicating when the chessboard pattern was detected in 3 or more views and was used to contribute to the calibration (green), and when it was detected in fewer than 3 views and did not contribute to the calibration (yellow). For Theia3D Apollo there is additional feedback in the form of a point cloud within the 3D view showing where within the volume the chessboard was detected and contributed to the system calibration. It’s important to note that it is not necessary to try to cover your entire 3D volume with these points (as you might with a wand calibration). However, this can be useful to review and allows you to identify positions within the volume where the board is detected or not, which may prompt changes to your technique, camera settings, or lighting conditions to improve the visibility of the chessboard in all positions.
Figure 2: Visual feedback provided after processing a chessboard calibration, showing the 2D detection zig-zag pattern on the camera views (left) and the detection point cloud in the 3D viewer (right).
Adjust Calibration Tool
One of the new tools introduced for Theia3D Apollo is the Adjust Calibration tool, located in the Calibration dropdown menu, which can be used to modify the position and orientation of your global coordinate system. This tool can be used any time calibration data is present, whether it’s immediately after processing a new chessboard calibration, or after loading an existing calibration .txt file. In either case, you can use the position and angle sliders to adjust the global coordinate system, which updates live in the 2D and 3D views. Once you’re satisfied with the new global coordinate system, you can apply the change and save the adjusted calibration as a new .txt file. This tool can be useful for cases where your chessboard calibration was unsuccessful in setting the origin in the desired video frame, or if you just want to relocate the origin to change the global positioning of the tracked people within the volume.
Figure 3: The Adjust Calibration tool (left) can be used to modify the position and orientation of your global coordinate system while maintaining the relative positions of your cameras from a new or existing calibration.
Video Metadata Tool
The Display Video Metadata tool located in the Tools dropdown menu allows you to easily review important details about the videos you have currently loaded, including the path to the files, trial folder name, video resolution, length of the trial (in frames), frame rate used to record the videos, and any offsets between each video file and the rest of the cameras. This last value will be <none> for most camera systems as the videos should be recorded with identical start and end frames, however Sony RX0 II camera users may see small values here since these cameras use timecode synchronization and the start frames may not be perfectly aligned.
Figure 4: The Display Video Metadata tool provides useful details about the currently loaded videos for your easy reference.
Enhance Videos Tool update
The Enhance Videos tool, located in the Tools dropdown menu, allows you to adjust the brightness, contrast, and white balance of the loaded videos prior to running analysis or processing a calibration. While it’s always recommended to try to capture bright, well-balanced videos at the time of recording, this tool is a lifesaver when the videos don’t turn out quite right. Until now, video enhancements were applied to all loaded videos, but for Theia3D Apollo it is now possible to make different adjustments to individual cameras! This is especially useful for adjusting videos from environments where cameras facing different directions are subject to vastly different lighting, such as cameras facing the sun or towards large windows. And, this should also make it easier to achieve strong calibration results for situations where cameras need to use different white balance settings to clearly detect the chessboard.
Figure 5: The updated Enhance Videos tool, allowing adjustments to individual camera views or all views at once.
User-Specified Default Preferences
We heard your feedback, and we’re happy to report that Theia3D Apollo includes the ability to save and apply user-specified default preferences. Gone are the days of manually adjusting your preferences every time, or copy-pasting a saved preferences file from one location to another. Now, you can modify the preferences, save them as a set of default preferences, and easily load those settings any time you are processing a trial within Theia3D or apply them within TMBatch. And with the ability to save multiple different default preferences files, you can select and set defaults for specific movement types (e.g. walk, run, countermovement jump) or projects, and never worry about loading the incorrect preferences again.
We’re especially excited about this functionality, which can be used within the Preferences pane in Theia3D or TMBatch. To save a new set of defaults, make your settings selections and click Save As Default to name and save a new set of defaults. To load a set of saved default preferences, simply use the dropdown menu and select from those available, then click Load Default. Simple as that, you can guarantee consistency in the settings used to process your data.
Figure 6: The updated Preferences pane allows you to Load and Save new custom preferences files, as well as Load and Save new default preferences. Select from your defaults using the convenient dropdown menu, and be confident that you’re always using the correct settings.
Moment of inertia to the output file
We have seen a few experiments where the moments of inertia are important - but we have required the user to continue to make measurements and input those manually in downstream software for them to be used. In Theia3D Apollo - we are using our rich 124 feature set to help estimate moments of inertia for both men and women, and exporting the radius of gyration for each segment. In the c3d file, this can be found in the Parameters section.
Figure 7: In the Parameters section you can find separate inertia settings for each segment. In this example, the pelvis is highlighted.
Display toggle buttons
Have you ever been examining tracked movement data in Theia3D and gotten tired of repeatedly using the Display dropdown to change what’s shown? Us too. So, we introduced display toggle buttons in the sidebar to make it a breeze to switch between display options with a single click.
Figure 8: Display option sidebar toggle buttons, to turn on and off (left to right) the person identification boxes, 3D segments, skeleton model, and segment local coordinate systems.
Here's a list of the many new features and updates we've discussed here:
Lens calibration visual feedback
Chessboard calibration visual feedback
Adjust Calibration Tool
Video Metadata Tool
Enhance Videos Tool update
User-Specified Default Preferences
Moment of inertia to the output file
Display toggle buttons
We hope you’re as excited as us for these new additions to the already robust Theia3D software. Check out our previous posts on Theia3D Apollo’s Updated Validation efforts, Model Definitions, and a discussion on Model Degrees of Freedom, and keep an eye on the blog for even more content on the new update coming soon! To learn more about Theia, click here to book a demo.