This is a question we often receive from prospective users. While markerless motion capture has opened up many doors that were previously inaccessible to other motion capture modalities, it is important to consider the challenges of moving your motion capture system from one location to another. For this week’s post, we interviewed support engineer Rob Kanko to understand if markerless motion capture truly is portable. Data referenced in this post was collected as a part of Rob’s master’s research and is also attributed to protocols developed by Jereme Outerleys (current Queen’s University PhD student). For more information on Jereme’s current work please check out his GitHub here.
Can you tell us a bit about yourself?
I’m a biomechanical engineer, researcher, and part of the support team here at Theia Markerless. As a mechanical engineering undergraduate student at Queen’s University I was excited and curious about the ‘Biomechanics Option’ offered as part of the program, which drew me in and exposed me to human biomechanics and biomedical engineering topics. After that, I had the pleasure and opportunity to work as a summer research student, then graduate student, and finally a research associate/lab manager within the Human Mobility Research Laboratory (HMRL) under Dr. Kevin Deluzio’s supervision. In combination, the cutting edge motion capture technology, interesting analysis techniques such as waveform principal component analysis, and my interest in human movement got hooked on biomechanics. Luckily for me, Theia Markerless was getting off the ground at the start of my master’s degree, and I ended up performing several validation studies on Theia3D for my graduate thesis. The promise of markerless motion capture and the new era of biomechanics research it has unlocked keeps me excited for the future of this field and excited to be part of the Theia Markerless team. Outside of my professional and research interests, in my spare time I enjoy running, hiking with my dog, and tinkering with bikes.
Can you tell us a bit about your experience with motion capture?
I’ve been a motion capture user for over 6 years now, which started with fairly comprehensive training in the HMRL on Qualisys Oqus (marker-based) camera systems, AMTI force platforms and instrumented treadmill, and wireless surface EMGs. The research context in the HMRL has long been on knee osteoarthritis, with a particular focus on knee unloader braces and analysis techniques such as principal component analysis and statistical shape modeling, to study the etiology, progression, and treatment of knee osteoarthritis. It was in this context that I learned a lot of the skills and techniques required to perform detailed biomechanics research.
Of course, after a few years of experience with marker-based motion capture I started on the validation studies with Theia Markerless, and soon became quite familiar with video-based motion capture data collection and simultaneous marker-based/markerless data collection using Qualisys Miqus cameras.
After the initial validation efforts, research using Theia Markerless continued briskly in the HMRL. We developed close collaborations with several orthopedic surgeons and advanced practice physiotherapists to start up several studies of clinical populations using markerless motion capture. Using markerless technology allowed us to easily recruit patients and perform data collections in a matter of minutes, which simply wasn’t feasible with marker-based methods. For this, we installed a new OptiTrack Prime Color camera system in order to record synchronized force data with the video cameras.
When did you take Theia3D out of the lab? What were the goals of this data collection?
Around the same time that the orthopedic population collaborations were gaining steam, we started taking our Sony RX0 II camera system out of the lab to start building a large dataset of healthy walking in the local population. One of the biggest challenges with performing research with human participants is recruitment, so being able to bring the ‘lab’ to where the people are instead of the other way around allows much larger samples to be collected more quickly. In doing so, we were able to quickly accumulate gait kinematics from a diverse sample of hundreds of healthy people. This dataset includes people of all ages in their own clothing and shoes, and in many different environments, capturing what can be considered a more realistic representation of healthy human gait away from the sterile lab environment.
What challenges did you encounter?
For our team, one of the biggest challenges was identifying suitable data collection locations that met our needs for space and had a sufficient number of passersby to recruit. Since we were recording gait data, we required a certain amount of space to allow participants to have multiple strides within the capture volume before reaching the other side, and a sufficient width to allow the cameras to adequately cover the area of interest. Typically, this required a space of approximately 15 m x 5 m, which although not enormous is still a sizable space to occupy. This is part of the reason these collections started in outdoor environments where space is less of a commodity and there was no specific need for approval to occupy the space. When considering indoor data collections, there were relatively few large enough indoor spaces and specific approval from a building manager is typically required to occupy these spaces.
Our second requirement, subject access and availability, was established so that we knew we would be able to gather enough participants to feel that we had not wasted a full day only to recruit a few participants. The effort required to undertake these out-of-lab collections usually felt worth it when we recruited more than 40 or so people. To this point, we found that high-traffic outdoor spaces on the university campus or buildings with a large atrium, and specific events such as science fairs were especially good opportunities for recruiting and generally had high participation rates. In one instance we also collected data in the warehouse of a local brewery!
What practical considerations should users think about when taking markerless motion capture out of the lab?
Taking a motion capture system for an out-of-lab data collection requires several practical considerations, including logistics and dealing with real-world factors. For most motion capture users, the lab space is the ideal controlled environment where the technology is set up optimally, so planning to perform research ‘in the field’ is a whole new ballgame.
Logistically, we had to consider things like how to transport the equipment safely and how many people are required to transport and set up the system. For these remote data collections we were using a system of 8 Sony RX0 II cameras, which are perfect for the purpose - small, light, durable, waterproof, with sufficient fields of view. We purchased a few items to protect the equipment and allow easier transport: a padded camera case to hold all 8 cameras, small ball-joint type camera mounts that are quick to adjust, backup batteries and memory cards with a carrying case, and a folding wagon-style cart to carry our 8 tripods with everything else piled on top. We used a projector travel bag that was lying around to organize the network switch and cables, and some reusable tote bags and bungee cords to keep everything else together and strapped down. Other items that we brought for planning and marking out the setup included tape (various types and colours), scissors, rope, markers, sidewalk chalk, a measuring tape, and a notebook. Whenever possible, we used the cart to transport all of the equipment and simply walked to our data collection location, but in some cases it was necessary to drive the equipment to the site. With a bit of practice, we were able to get the setup time with 2-3 people down to 20-30 minutes, with the longest task being optimizing the camera orientations for the new space. Taking down the setup is even faster, usually about 10 minutes total.
Besides transport, we required an available power outlet. These are sometimes harder to locate at outdoor locations, but having a long extension cord (100 feet or more) can provide quite a bit of flexibility. Since the only power requirements are the network switch and the laptop used to control the cameras, a long extension cord with two plugs at the end or an additional power bar is sufficient.In some instances, we also tested larger camera setups using portable power sources including a gas generator and electric power bank.
We also made sure to bring some signage along with us. We had a few sheets which gave notice that video recording was in progress for transparency and awareness of people nearby, and a couple other signs to try to entice potential participants. A large picture of a skeleton overlay from some markerless Theia data goes a long way in getting people’s attention.
One final consideration to make when planning out-of-lab data collections is weather. A calm, clear day is ideal, and cool temperatures are generally better for camera and computer equipment. While the cameras are water resistant, most of the other equipment is not, and so avoiding rain or snow is important. And wind can be especially challenging, as it doesn’t take much to cause the cameras or tripods to shift, requiring vigilance when it comes to calibrating regularly (which you should do for out-of-lab data collections anyways). Finally, it’s crucial that the calibration chessboard is flat to ensure high quality calibrations, and certain conditions such as abrupt changes in temperature or humidity can cause warping. So, keep an eye on your calibration board and do what you can to mitigate these effects.
If you decide to take on out-of-lab data collections, remember to take careful notes to learn from your experiences and mistakes, and try to incorporate some fun or outreach while you’re at it! Bringing research out into the real world is a great way to engage with the community and spread interest in biomechanics and STEM in general, so make the most of the field trip.
Example product links:
To learn more about Theia, click here to book a demo.