“Who’s Who” in Virtual Reality: The Power of Machine Learning

What would you think if I told you that a Virtual Reality (VR) system can know who you are just by watching how you move your eyes, hands, and head? It’s not magic, it’s machine learning.

VR user identification is about recognizing you among others every time you jump into a VR experience. And it’s not just for fun! User identification has many possibilities in VR, such as making setups like classrooms or gaming lounges more secure and personalized. In learning experiences, VR with user identification can optimize lessons!

Machine learning is in charge of making all this happen. First, it needs a dataset (a lot of organized data). This will be its source and the basis of its analysis. And it’s not just any analysis! In VR, you can have a dataset of many users with data from their eye movements (remember, this is possible as discussed in other blog posts clue: eye tracking), as well as head and hand movements. Each movement you make, considering your height, speed, rotations… is unique to you. The algorithms (e.g., Random Forest) do their work by spotting patterns between sessions and across the entire dataset to identify you accurately.

You might not immediately see the usefulness of this technology, but imagine two players sharing the same VR headset. The game would adapt to the first player, and it would be a hassle to reconfigure or personalize everything for the second player. With user identification, this problem is solved: you play, you are identified, and everything is optimized for you! Even your stress levels can be detected, making the experience easier and more enjoyable.

As researchers, we studied the effectiveness of this approach. That’s why we have something meaningful to share. The research recorded over 50 million data points from 64 users across multiple sessions using a VR overhead crane simulator. That’s a lot of movement to track! The results? Machine learning can effectively identify users, especially as they become more familiar with the tasks and show more individualized behavior. It’s a bit like fingerprinting but with your body and eyes instead of ink. You can discover more in our research: paper link.

The good things also come with some concerns. You might worry about the uniqueness of your eye movement behavior because now we know it can identify you. And that’s true. So if you want to try it, make sure your data is protected by reading the disclaimers carefully and understanding how your data will be handled. Additionally, make sure your data is anonymized.
Have you ever thought about how VR could recognize you next time you play? Would you be amazed or creeped out?

#VirtualReality MachineLearning UserIdentification iVR SeriousGames Education ImmersiveLearning DataScience UserExperience Innovation