This expert says VR’s biggest problem is how we’re using it

Despite high-octane events in the augmented reality (AR) and virtual reality (VR) landscape, such as the launch of Apple’s new $3,000 headset, the day-to-day reality is that the sector is still quite niche. Headsets are pricey and clunky, wearing one for more than 30 minutes often produces motion sickness, and there are few games to offer truly memorable experiences. 

The big question is whether VR is destined to remain a fringe technology, or are there better, more appropriate uses for it?

Also: Will Apple’s headset signal the beginning of the immersive internet?

ZDNET spoke with Jeremy Bailenson, founding director of Stanford University’s Virtual Human Interaction Lab and a pioneer in the field of AR/VR. Bailenson’s career started off in cognitive psychology before he moved on to study how virtual experiences lead to changes in perceptions of the self and others. 

His lab builds and studies systems that allow people to meet in virtual space, and explores changes in the nature of social interaction. He also has a company, Strivr, whose Immersive Learning Platform has trained Fortune 500 companies — such as JetBlue and Walmart, as well as municipal governments and nonprofits — in an effort to transform training, education, environmental conservation, empathy, and health. 

Also: Apple Vision Pro first take: 3 reasons this changes everything

Here, Bailenson discusses why VR companies might be chasing a flawed strategy of positioning AR/VR for entertainment and communication versus building unique experiences that tackle society’s intractable problems. Six of his VR documentary experiences have been official selections at the Tribeca Film Festival. His lab has also exhibited VR in hundreds of venues, ranging from the Smithsonian to the Super Bowl.

ZDNET: Tell us a little bit about yourself.

JB: My PhD is in cognitive science. I defended my dissertation on building mathematical models of how the mind works. This involved running experiments on humans and building a low-level AI to try to study how one can represent thoughts, decision-making, and reasoning. I just decided at that point, I just don’t want to be a cognitive scientist. The content wasn’t grabbing me as much as it should have.

At the time, I was reading a novel called Neuromancer, a science fiction novel that really just kind of blew my mind about how you can think about building these experiences that may not be intelligent, but are certainly perceptually persuasive. It redefined what it means to be a human in an age of technology. And so, in part, because of reading that book, I decided to try a different route. So, I got a postdoc at the University of California, Santa Barbara. I was there for four years learning how to build the hardware, and eventually made my way to Stanford and the department.

ZDNET: What were some of the things that were said about AR/VR in terms of it being a new world with new possibilities?

JB: The narrative was very different back then. We weren’t thinking about consumer-grade VR, we were thinking about VR as a tool to study. First of all, I was in a psychology department. So, my postdoc involved using VR to study realistic simulations and see what one would do in a social situation.

Also: Best VR headsets right now: For gaming and the metaverse

It was very much about using it as a tool. Back then, VR was more like an MRI machine — it filled up a whole room, cost seven figures, and needed a dedicated engineer to run it. It was less about engagement, entertainment, communication, and education as we think about it now, and more about “this is a very special thing that we’re going to study in and of itself.”

ZDNET: But everybody else has jumped on the entertainment bandwagon now, haven’t they?

JB: Yes. It’s pretty striking. In 2022, there were 10 million headsets sold and 90% of them were sold by social media companies like Meta and Byte Dance. Their model is engagement and entertainment, and some communication. But having the hardware driven by content platforms — it’s a bit of a change for the medium, historically.

ZDNET: What do you make of the new Apple VR headset?

JB: I love that they built a crushingly good piece of hardware with really low latency on the video pass-through and eye tracking. This allows it to do calibration really quickly and easily, and the images are crisp. But the challenge of VR has never been about tech. It has always been about the use case and the experience you want someone to have in VR.

ZDNET: So, how is VR being used today? 

JB: Currently, if you go to a Best Buy, you can find many headsets for $300. There are millions of Americans who have played a video game in VR this week. However, I like VR for solving hard problems, fixing things that are broken. And one thing that’s not broken in the US are video games.

Also: The best PC gaming headsets

Arguably, the thing that we do better than most countries is making video games that are really good, so much so that they don’t need VR. The question is, when do you need VR? 

ZDNET: So, is this a business strategy that is flawed?

JB: It’s not for me to say that their business strategy is flawed, because I’m not qualified to understand macro trends, and, of course, return on investment for big companies. However, I’ve spent a lot of time working for big tech companies. 

Also: What is the metaverse, and who will build it?

I was with Samsung for five years, and worked directly with the president of Samsung and so I am familiar with companies that think about their big-picture strategy. And I’ve always said that the challenge for ROI for the big tech companies is that one can’t make money based on minutes used of this device.

It’s not gonna work because it’s an intense medium. And even if you can chip away at the small things that can cause simulator sickness, things like latency, or how many stereoscopic layers you’re going to be able to render on a physical headset, or how many focal points are on it — those are going to help a little bit. But the notion that people want to wear something on their head for a long time is just not something I’ve seen hunger for in my time using the medium. And you know, we think about ROI with VR in very different ways. 

ZDNET: Can you explain what you mean by that?

JB: You’re familiar with my DICE acronym for VR? Things that are Dangerous, Impossible, Counterproductive, or Expensive? Let me illustrate further using my Walmart active shooter training module.

I co-founded a company called Stryvr through which we do VR training, and one of the most incredible and brutal things I’ve ever done was an active shooter drill we did for Walmart, one of our big clients. 

Also: Are VR headsets safe for kids and teenagers? Here’s what the experts say

When an active shooter comes into your store, there are a lot of things that you’re trained to do. How do you position your body? How do you talk to your fellow employees and customers to keep them safe? Where do you look? Previously, there was no great way to train people. We were using PowerPoints. 

Now, you put on the headset. It’s one of the scariest moments of your life. This guy walks in with a gun and starts shooting. So, the training is about 20 to 30 minutes. And you know, at every point we have you stop and make a decision. Then we give you feedback on the decision.

For instance, at some point, we’ll stop you eight minutes in and say, you know, we told you before you started that you shouldn’t look the shooter in the eye. And they say, “No, no, I didn’t do that.” So, we play back the heat map of their gaze in the scene and are able to point out that the employee spent 40% of the time looking the shooter in the eye.

ZDNET: How was that received?

JB: When the horrible event occurred in El Paso where the shooter came in and killed a bunch of people, a number of associates who were on the floor that day had been trained using our system.  And the CEO of Walmart says very clearly that lives were saved that day because of that training using our VR simulation.

ZDNET: Tell me about one of your most well-known projects, ‘1,000 Cut Journey’.

JB: We spent the last eight years building this 15-minute piece with Courtney Cogburn, an associate professor at Columbia University and an authority on cultural racism in the media, and its acute physiological, psychological, and behavioral stress responses in affected populations.

Also: What could Apple’s VR headset possibly do to justify its $3000 price tag?

It’s an experiential piece where you become a black male, Michael Sterling, and experience firsthand what it is like to walk in his shoes today. You first experience a racial bias as an elementary school kid in class through disciplinary action, then as an adolescent encountering the police, and as a young adult experiencing workplace discrimination. It premiered at the 2019 Tribeca Film Festival and now anyone who owns a Meta Quest headset can download ‘1000 Cut Journey’ for free and experience this educational piece about systemic racism.

ZDNET: Tell me about your project about the homeless population in San Francisco.

JB: It is a piece that puts you in the position of an individual who cannot get housing for whatever reason and who has become homeless. The project was sponsored by the Robert Wood Johnson Foundation. They gave $400,000 to fund a team to work on the project and helped us work with homelessness organizations. 

When I first started the project, I was trying to figure out what we could do to help the homeless most, and the answer, which was not obvious to me then, was that we need to build more affordable housing. That project has had a lot of impact when we brought it to various government or nonprofit events. I’m proud of that piece, it’s been used by lots of different scholars who wanted to understand when VR works for empathy.