You've seen the photo. Facebook CEO Mark Zuckerberg striding through Samsung's Mobile World Congress press conference, breezing past dozens of reporters and industry types too engrossed by their Gear VR headsets to notice the most powerful man in the Internet. Zuck's smile suggests he can't quite believe the scene. It's an instantly iconic image, something that will make an alien race one day wonder what the hell those strange beings on Earth were doing.
It's also incredibly revealing.
When Zuckerberg reached the stage, he waxed poetic on the social possibilities of VR. "Pretty soon," he said, "we're going to live in a world where everyone has the power to share and experience whole scenes as if you're just there, right there in person." The press ate it up. But go back to the people in the audience. They're at Mobile World Congress, the annual smartphone extravaganza in Barcelona, Spain. They're delirious from announcements and presentations. They're experiencing the future through a smartphone strapped to their face. And they're loving it.
For all the attention lavished upon the Oculus Rift and HTC Vive and Magic Leap, most people will experience the wonder of VR with the same device they use to send a text and summon a car. "While mobile VR may not trump PC-based VR gaming," says Nick DiCarlo, Samsung's head of immersive and VR products, "we do believe mobile VR can be the best when it comes to videos and social interactions." This makes sense when you consider two things. First, no other tech approaches the smartphone's scale: 6 billion people will own one by 2020, according to one report. And second, phones are relatively cheap. Even Oculus founder Palmer Luckey concedes that a Rift for everyone remains years away.
Meanwhile, smartphones are approaching the Rift's capabilities. Throughout the industry, virtual reality is driving the next generation of features and technology. Your next phone will be a kickass virtual- and augmented-reality machine. This new design focus will change everything about smartphones, from the chip to the chassis. The first wave of smartphones led us all far beyond calls and texts. The next will take us into entirely new worlds.
The key to good VR is processing power, which explains why the Rift and Vive require monster PCs. Qualcomm, which builds processors for just about every high-end smartphone, believes it can pack it all into the the chip in your phone. The company has spent years thinking about this. "It just became apparent," says Tim Leland, VP of product management, that a headset "was going to be a better way of getting information into people's visual cortex than having to rely on just tap-tap-tap." That realization prompted conversations with Google and others about the possibilities for VR before its engineers started exploring how to make those experiences work.
Qualcomm's engineers discovered that VR-capable chips require massive upgrades everywhere else. You need exceptional graphics processing, and the ability to power super-high-res screens. "You could argue it's overkill for a smartphone, but it's not when it's close to your eye," Leland says. You also need three-dimensional spatial audio and really powerful motion tracking. Most important, you absolutely need some way of dealing with several orders of magnitude more information without draining the battery in eight minutes. These are not small tasks. It took a few years, and many iterations, to get them right.
Those improvements are the core of the Snapdragon 820, Qualcomm's new flagship smartphone processor. The HTC 10, Samsung Galaxy S7, LG G5, and most other high-end phones already use it. The processor supports high-res screens, 360-audio, cameras with computer-vision abilities, and more. Qualcomm released an SDK so developers can take advantage of all this, plus the software minutiae involved in VR. DiCarlo says it's already made a difference. "Comparing our Gear VR Innovators Edition and the consumer Gear VR that we launched in November, the improvements in phone technology even over the course of a couple of years made for a much more comfortable, much more enjoyable experience." The idea is that these tools eliminate the need for a separate or optimized device. VR can be amazing even on a bare-bones phone.
Google makes one of those bare-bones phones, and in case you hadn't noticed, is beyond committed to mobile VR. "If you can afford a smartphone," says Mike Jazayeri, product director for VR, "then you should be able to get a great VR experience." Sure, the Rift and Vive are more powerful. "But don't underestimate what those [mobile] chipsets are doing. And not everyone needs all those polygons."
Because Android almost certainly will remain the world's dominant mobile OS, Google plays an integral role in VR. The Wall Street Journal said earlier this year that Google is developing a version of Android to handle the technology. Watchful developers spotted references to "VR Listener" and "VR Helper" apps in early versions of Android N.
Elsewhere within Google, engineers are hard hip-deep in virtual- and augmented-reality technology that could be as essential to a phone as GPS. That's not my line, by the way. It's from Johnny Lee, who leads Google's Project Tango team.
Tango is like a sci-fi-technology bingo board. Its camera array uses computer vision, depth sensing, and motion tracking to map the world around you, letting you do everything from navigate indoors to play augmented-reality games where the bad guy literally hides behind your couch. This kind of inside-out tracking means you don't need big sensors to see where you are. And that means truly mobile VR: Your phone uses its bonkers camera array to knows where you are and where you're going.
Like all mobile tech, Tango is improving quickly. The first developer kit, released in 2014, was a huge, hideous tablet that cost $1,000. One year later, it was smaller, prettier, and half the price. The first Tango Phone arrives this summer, from Lenovo. Jeff Meredith, a Lenovo vice president and the GM of the tablet team, believes it's much more than a tech demo. "I truly believe that the technology can be one that is totally pervasive," he says.
Lenovo started toying with Tango early last year as the company considered new ways of pushing smartphones forward. When everyone has fast processors, good screens, nice cameras, and Android, what do you do next? The answer is VR.
Lenovo's team wanted to get out in front. But it wanted make VR feel like a natural and obvious part of your phone. "You don't say, 'Man, I really need to initiate the backup camera on my car,'" Meredith says by way of example. It's just there. Lenovo wanted something like that. The company surveyed its options and realized Tango "was the only real viable technology and technology partner that could do what we aspire to do." So it worked with Google and Qualcomm to optimize a phone for Tango's technology.
The big challenge was cramming the Tango camera into a pocket-sized device, which meant redesigning the internal structure and figuring out how to keep everything cool. It also meant finding a way of making the device look nice, and not like an enormous insect staring at you from your desk. The result is a 6.4-inch smartphone---big, but not comically so. Meredith says 6.4 inches felt like the right balance of usability and capability. "That's not to say we won't go smaller than that going forward, or bigger than that," he continues. "There's tons of applications for all different screen sizes."
Right now, Lenovo's pondering three broad applications for Tango. The first (because it's the easiest for people to understand) is indoor navigation. Google Maps gets you to the museum, Tango tells you about the painting 36 feet to your right. Gaming's also a big one---Lenovo's even launching the phone with some accessories to improve the experience. The last part is something Meredith calls "utilities." He means the ability to map your living room and then superimpose in your new couch, just to see how it looks. It's a lot, but it's just the beginning. Lenovo's launching an app incubator for developers, and as Tango grows the tech's capabilities surely will too.
Like most mobile VR and AR tech, Project Tango is early. And it's waiting on a lot of necessary tech, like 5G networks for lightning-fast streaming. The industry is switching from LCD to OLED displays, which refresh faster and look better. Headsets need better lenses, less weight, and greater comfort. Such things take awhile, and until then the experiences will fall short in obvious ways. "This is V1," Evans said. "This is the test and the demo that make you think this is going to be interesting, but it's not there yet."
What matters here, though, is the idea has caught fire. Big players are committing to mobile VR. Intel's RealSense technology features many of the same ideas as Project Tango, and Intel has released a developer kit phone with both. (It has, um, a lot of cameras.) Nvidia's designing graphics chips with VR in mind. Lenovo, LG, and Samsung have headsets that are cheap and even free add-ons when you buy a new phone. These things are pushing VR into the mainstream. That, in turn, amps up the pressure for manufacturers to build phones that can do even more.
Your next phone may not be an incredibly realistic window into virtual reality. But the one after it almost certainly will be. And when it arrives, it will democratize VR in a way no headset tethered to a PC ever could.
