Mark Zuckerberg has been talking a lot about augmented reality and investing a lot, too. Yesterday he did it again in an all-hands meeting that was live streamed to his Facebook account (it was very interesting to watch, by the way). Everyone in Silicon Valley knows that Facebook is spending billions on a future augmented reality headset and operating system, many even point to some new buildings it is building and tell me “that’s where Facebook is going to put its AR team.” Facebook already has internal plans for the next five years and most of these new plans, and excitement, are about augmented reality. It isn’t alone, we know of many companies that are spending billions on same, including Microsoft, Google, Amazon, Sony, Magic Leap, Huawei, and others. The tech industry is gearing up for a new paradigm shift.
At its recent Oculus Connect conference, where it was speaking to virtual reality fans and developers, Facebook made it very clear during its keynote “we are building AR glasses.”
The problem is that saying that confuses the market. Is VR going to turn into AR? Will VR “die?”
I’m responsible for that, too, having co-written a book “The Fourth Transformation” talking about how, in the future, we would “move through” computing while wearing said glasses. I now call the whole field “spatial computing” which includes all computing humans, robots, or virtual beings move through, including augmented and virtual reality.
Note that I’m not talking about the confusion that comes from what the industry is struggling to call next-generation augmented reality. “Mixed reality,” “extended reality,” or something else that some marketer will come along with.
I’ll leave the naming war alone for now. It’s clear there are two separate camps. One VR, one AR, and that will become clearer over time, and trying to force the two together, like Zuckerberg and I were doing is causing massive confusion as we wait for Apple, and others, to ship usable augmented reality devices.
VR is when you see nothing but virtual things. AR is when you see virtual things on top of, or replacing, parts of the real world.
While that belief isn’t wrong, where both Zuckerberg and I have it wrong is hinting that VR will “morph” into this kind of “mixed reality” future where VR and AR will merge and you’ll do both with one device on your face. I’m not alone. Almost everyone in the industry has “AR/VR” listed on their Twitter account. Most of the media that covers VR also covers AR. Almost everyone is hinting the two are joined. Many people in the industry talk as if it is a given that the two will be joined soon.
We are all wrong.
After seeing tons of bleeding edge optics and displays you soon will be wearing on your face, and thinking through the problem, talking with many VCs, big company employees, entrepreneurs, and developers, I realize I made a huge mistake in believing the world would soon be “mixed.” That won’t happen anytime soon. Soon being less than five years.
Why? The two use cases are totally different.
VR is about immersion. Escape from the real world.
AR is about utility. Adding utility to the real world.
The two have completely different goals and will continue to need completely different devices for a long time. At the high end of VR you can plug your headset into a very large and very powerful GPU from, say, NVIDIA. In fact, Facebook announced such a capability even for its standalone Oculus Quest. Truth is that the tiny GPUs in mobile phones simply aren’t good enough for many workloads especially if you are trying to “immerse” your user in, say, a bleeding edge video game.
Also, if you are trying to get high-resolution, wide field of view monitors on someone’s face, you won’t be able to drive those from mobile phones and their small GPUs anytime soon.
You can see this in today’s AR products (and despite what Microsoft or Magic Leap call their devices, they really are augmented reality devices that you wear on your face). The field of view in these devices is about half the field of view in your average VR device. They do NOT do immersion, or, if they do, they don’t do it well. Immersion is the feeling you get when your brain believes you are “inside” and “experiencing” something. When you go to a football game you are “immersed” in a way that you can’t get by watching the same game on TV. That feeling is what VR is aimed at giving you and it delivers it very well. So well, in fact, that when I first experienced VR I experienced vertigo. I’ve never experienced that while watching TV or looking at my computer screen.
This all snapped for me when I met Jon Griffith in the hallway at Oculus Connect. He was carrying around a VR headset. He asked “want to go to Everest?” He carried dozens of cameras up the highest mountain in the world and recorded Sherpa Tenji climbing it without oxygen.
The experience of being inside Griffith’s VR experience was gripping. It “felt” like to me that I was on Everest with him, and not in the San Jose Convention Center. I was so immersed I forgot about the crowds swirling around us and felt the cold and fear as he took us across ladder bridges and up steep cliffs of ice.
You can watch a trailer of his experience on YouTube:
VR is different than where we are going with AR and will not be replaced by AR because of that feeling of immersion.
Here’s the problem that’s coming soon.
Back to the augmented reality devices that Zuckerberg is promising. These augmented reality visors are designed to be worn in the real world and “augment” everything you look at. Qualcomm told me that by the end of 2020 we will see a ton of them announced (Qualcomm makes the core technology that Facebook’s Oculus Quest uses. Literally everyone in the industry other than Magic Leap and Apple uses Qualcomm’s spatial computing tech stack).
The thing is, these new “wearable in the real world” devices have a completely different design set of tradeoffs. They must “look cool.” In other words, they must at least look like a pair of Oakley blade sunglasses. They can’t look nerdy like Google Glass did. They certainly can’t be big and ugly like Oculus Quest/VR headsets are.
They also need to be battery efficient. You don’t want to carry a huge battery on your head. And they need to have displays that show you the real world. You will NEVER wear VR if you are putting your hands into, say, a bandsaw. Too dangerous. You also will never use a VR headset while crossing a street, even if that headset could magically become 1/10th the size of today’s headsets that are big, black, and ugly.
The problem with showing you the real world is you just can’t make an optic, yet, that gives you great immersion and great virtualized monitors. Maybe a decade from now that will happen, maybe even five years from now, that’s possible, but I think I will own two separate devices for a long, long time:
- A VR headset for personalized media and experiencing “immersion.”
- An AR headset for augmenting the real world and using at shopping malls, school, work, while driving, etc., focused on “utility.” Or, focused on being social with other people (Facebook’s plans for 2025, for instance, include you playing a new kind of virtual frisbee or football game on top of the real world, while at a park, for instance).
It is a mistake to think that VR will stop being interesting five years from now just because augmented reality is a bigger market and even though you can do a lot of 3D things, including playing new kinds of games on top of the real world these will hardly be “as immersive” as a full-blown VR headset hooked up to a gaming-level PC. I’m sorry I made that mistake and will keep that in mind, even though Irena Cronin and I (and our work for clients at our research/consulting firm Infinite Retina) are far more excited by the market potential of augmented reality visors (or, glasses, if that’s what you want to call them, I like thinking of visors because the first devices will struggle to get to the size of the glasses I’m currently wearing).
Now, the deeper question is: will software developers see them as separate, or the same?
We built a Twitter list of about 800 developers in spatial computing (both VR and AR) and are meeting regularly with developers (another was at my house this morning) and you can clearly see a separation in skillset evolving. Some, like Aidan Wolf, are clearly focusing their lives on utilities for augmenting the world. Most others are focusing almost wholly on immersive experiences. Watch this list for a while and you see there is a real difference between people building for AR and those building for VR.
One of the most successful VR developers and that show this demarkation between AR developers and VR ones is STRIVR, who makes VR training for Walmart. Notice how VR is used and why immersion is so important to training. This isn’t about augmenting the real world, it’s about immersing you into the training so you remember what you’ve learned.
Yes, they use similar tools, if not the same tool, as AR developers use: like Unity. Yes they think in 3D.
On the other hand who think more in AR are much more adept at sensor fusion and computer vision techniques. Why? Because they are trying to augment the real world, while those focused on VR are better storytellers, generally, and are more focused on techniques that immerse viewers into a game or media of some kind, like Jon did with his Everest VR experience.
Zuckerberg and I are messing up because we are trying to put both kinds of developers together in an unnatural marriage. For now we should stop it, and I will.
Myself, I’m going to focus more efforts on AR developers like Aidan (even to the point where I’m starting to build new lists of those just focused on augmented reality instead of VR). Why? Irene and I are much more interested in utility and augmenting the real world than, say, Hollywood movies, or video games. I respect those a lot, and love experiencing them on my Oculus Quest, but my professional life will be far more focused on, say, a surgery team that’s using augmented reality/spatial computing to augment their jobs, or a car manufacturer that’s using augmented reality to augment theirs.
Those who are working on immersive media and games, though, deserve to know their work is valued and that they have a strong future, and Zuckerberg, by having Facebook brag about its work in augmented reality without putting it into proper context, and explaining that VR work will remain separate and important for a long time makes them wonder if they have a future and, I can say now that, yes, yes they do have a future and a wonderous one at that. There are tons of new VR products coming and the customer base will continue to grow (albeit at a slower pace than I was expecting to see in 2019 for a variety of reasons).
Even if someday soon we have a huge optics breakthrough (and I’ve seen some that are possible in, say, 2023 or later) I think we still will see innovations in VR-only approaches that will bring better immersion. We should stop pushing the industry to join these two very separate use cases together into some sort of frankenstein device. We should stop forcing developers to be joined in some sort of weird way just because they both use the same tool, Unity or Unreal Engine.
If we do, we will always come up short. A small device probably won’t have great immersion and a bigger, passthrough device, that has amazing immersion won’t be all that great for wearing around the real world.
By forcing them together we will have to compromise and that’s not good for either camp at this point. Virtual reality should be seen as very different than augmented reality, even if both use the same spatial computing grouping of technologies. Zuckerberg should separate the two into separate strategy lines and explain these differences in a much clearer way, at least for the next few years.
The problem with reacting to Apple
Here’s another way to think about it. Apple hates VR, we’ve been told, and sees it as unsafe and unmarketable. It might be right about that, and I believe that’s one reason why Zuckerberg is signaling that he, too, is working on AR. He doesn’t want Apple to be able to position Facebook as “lame.” (Let’s leave aside the fact that Facebook has caused a ton of positioning problems all on its own, thanks to its various privacy and security missteps).
The problem with that is Zuckerberg is actually weakening his own hand that he’s spent billions building: VR and its immersive capabilities. Facebook has spent billions acquiring, and building, its Oculus brand.
Instead of trying to widen this brand to include augmenting the real world, Facebook should make Oculus stand for immersion and keep it only for VR. Yes, Apple will attack VR as “unsafe” and “not useful.” Zuckerberg should reply “yeah, but AR can’t immerse you.” If Oculus stands for immersion, it’ll win. If Oculus tries to play both sides of this coin, it’ll appear weak and confused, in comparison to Apple’s AR-only approach.
It’ll also confuse and piss off developers. There are tons of developers who want to exploit immersion. Andreessen Horowitz, er, A16z, invested tens of millions in Sandbox VR, and Disney has done same for the Void, for instance. These companies make very fun, very immersive experiences you play at your local shopping mall and both are expanding fast. They never will do AR. Why? They are totally about immersion and helping you escape from the real world.
Facebook should keep the Oculus brand focused totally on immersion, while letting its new “Horizon” brand focus on social interactions in virtual and augmented reality, and I recommend it come up with a new brand for its augmented reality devices and keep those separate from Oculus to keep from confusing the marketplace.
Doing that will let them have a stronger position against Apple, and others, like Microsoft, and Magic Leap, who don’t yet have a play in virtual reality, and probably won’t since both of them have decided to do an augmented reality strategy only for now.
Facebook should strengthen its VR brand, not weaken it. Immersion should be the rallying cry. If Facebook can get everyone to understand immersion is magic, then it’ll withstand any attack from Apple or Magic Leap. Plus, that’ll give Facebook the credibility to start up a new brand around the utility of augmented reality and really go to war with Apple, amongst all the others that will soon sprout new spatial computing products.
I will stop weakening VR. It’s huge and important and won’t be killed by AR.
Will Zuckerberg get this too? Time will tell.
Scobleizer 