Tracking in XR: What Seems Trumpery, But Truly Decisive

Tracking XR hardware is the first thing you should tackle when preparing an experience for users or debugging your application. You might think the most important thing is to show off the app and ensure it works perfectly, but guess what? It’s surprisingly easy for things to go wrong! Why? Mainly because of the tracking of the HMDs and the setup of the real environment (don’t worry, it’s not a huge problem with your programming skills or logic—at least not while you’re reading this post).

First off, you need to know that there are a ton of factors that can mess with your tracking! These can also affect how the application runs and how well interactions work, so keep both in mind.

Secondly, set up your tracking zone according to the hardware and software instructions. But here are some extra tips that can save you a lot of headaches:

• Lighting: This is a biggie. Direct lighting can overload the sensors because intense light can create “ghost” objects that aren’t really there. Some HMDs, like the Meta Quest, use cameras to track the user and environment by processing light reflections. When there’s a really bright light source, like the sun or a powerful lamp, it can create too many reflections, leading to fake objects or making it hard to recognize real ones. Even with laser-based systems like the HTC Vive Pro, too much light can interfere. So, don’t use HMDs under direct sunlight or near a window without blinds!
• Darkness: Extremes are bad here too! If it’s too dark to recognize objects, the tracking will be way off.
• Patterns: Some HMDs use cameras to recognize objects, but what if the patterns or forms are too repetitive? The cameras can get confused, making it tough to perfectly position the user. Avoid repetitive patterns or plain surfaces; adding some stickers on the floor and walls can help a lot (and it’s even better if they’re different shapes).
• Cleanliness: It might seem obvious, but dust can mess with your tracking. It’s important to keep your equipment clean so the cameras can see the objects or the laser bases can send out signals properly. Clean off fingerprints too! Use microfiber cloths (the kind you use for glasses or screens) and never, ever use alcohol, other chemicals, or paper towels. They can damage or scratch the hardware! For tough spots, use a bit more cleaning fluid (no alcohol) and a microfiber cloth. Clean gently in circles without pressing too hard. And remember: the cleanest setup isn’t the one you clean the most, but the one you keep from getting messy in the first place. So, store your HMDs and bases when not in use.
• HMD Fit: Make sure the HMD fits well on the user’s head. If the user wears glasses, make sure they’re clean, small, have slim frames, and ideally, are anti-reflective.
• Placement: Proper placement of the play zone is crucial. This also applies to the bases: don’t stand in front of the sensors! Real objects should be static, and the user needs to be fully visible to the sensors. Avoid obstacles, and place the bases on opposite sides, looking at each other, with a clear line of sight to cover the entire play zone. Elevate them to track as much of the area as possible.
• Avoid Vibration: Vibrations or impacts can shift your base or sensors, mess up tracking, or even damage the internal hardware, leading to worse performance.
• Controller Tracking: If you lose tracking of your controllers, it might be because they’re not properly visible. Make sure they’re within the play zone and seen by the sensors. If the issue persists, restart them (turn them off and on), make sure they’re charged, and check that they’re calibrated.
• Frequent Calibration: Regularly recalibrate everything, especially if something has moved—even slightly! Marking where the base or play zone limits were on the floor or wall can make recalibration easier than just relying on photos.
• Restart: Shutting down the HMDs and trackers regularly can help reset the system and clear out any tracking issues. It’s a simple trick that often fixes problems.
• Software and Firmware Updates: Make sure everything is up to date. Developers often fix tracking issues in updates that other users have reported.
• Update GPU Drivers: Poor GPU performance can mess with the whole tracking system. Keep those drivers updated!
• Interference: This can happen if multiple users are close to each other, so either avoid it or separate the play zones well. Also, if you’re using different bases in separate play zones, make sure they’re on different channels to avoid them interfering with each other. Other signals—like from smartphones, Wi-Fi, and Bluetooth—can also cause interference, especially with non-laser-based systems. Keep those devices away from your HMDs.
• Hardware Issues: Finally, consider that your HMDs, bases, or sensors might be faulty. If none of the above tips work, you may need to check for hardware problems.

Now it’s your turn to put all these tips into practice! XR tracking is a trial-and-error process, so be patient. Sometimes things that were working perfectly before just stop working, or vice versa! Got any additional problems or tips? Let us know in the comments!

#Tracking #XR #VR #AR #MR #VirtualReality #ExtendedReality #HMD #Lighting #AugmentedReality #Calibration #Interference #Hardware #Software