Interview: Microsoft HoloLens 2 Team Speaks — Inside the Specs, Design, & Future of Enterprise Augmented Reality

As I've mentioned in recent days, it was refreshing to discover that Microsoft's HoloLens 2 stage production was every bit as accurate as presented (check out my hands-on test of the device for more on that).

But beyond watching Microsoft's deft HoloLens 2 stagecraft, and trying the product out myself, it's important to understand some of the thinking that went into a device that many have been waiting years to get their hands on. To that end, I met with Microsoft's Greg Sullivan, the company's director of communications, who devotes much of his focus to the HoloLens and the Azure cloud computing platform.

• Don't Miss: HoloLens 2, All the Specs — These Are the Technical Details Driving Microsoft's Next Foray into Augmented Reality

In the past, Sullivan and I have talked a lot offline about immersive computing and what the future holds for Microsoft, as well as the overall space. This time, we covered a number of topics, including nit-picky issues with the original HoloLens, to the future of augmented reality as a computing platform in general.

Although I've already stated that I think the HoloLens 2 is now the best augmented reality device you can buy, the following discussion reveals that, as powerful as this device is, we're still just at the very beginning of a great new computing era, and there's so much undiscovered immersive computing territory to explore.

Next Reality: So we're here at the biggest wireless tech conference on the planet, but there's no cellular modem in the HoloLens 2. Some early hints indicated that this might be the case, and it was something some were hoping for.

Sullivan: The chip is the Snapdragon 850 compute platform, we worked with Qualcomm on it. But yes, the SoC (system on a chip) does not have a cellular modem ... It's not happening in this generation.

Next Reality: The hand tracking on the HoloLens 2 is brilliant, but can you explain why your team decided not to include a hand controller for more delicate and specific functions in the update?

Sullivan: Well, you know we had the clicker in the first HoloLens [a small, thumb-sized controller with one button, which isn't included with the HoloLens 2]. What we're focused on this time is direct manipulation [of virtual objects], but nothing else to announce.

Next Reality: Can you talk about why Microsoft isn't adopting the Snapdragon XR1, which is Qualcomm's first chip its first chip dedicated to AR and VR?

Sullivan: We have a significant amount of IP invested in our custom silicon that we develop. First for HoloLens, the previous generation, in what we call the HPU [holographic processing unit], it's an application-specific integrated circuit coprocessor that offloads a lot of the environmental awareness, mapping, and other work that would otherwise tax the application CPU. So the application processor can handle the applications, the HPU can handle a lot of the environmental understanding work and all the data that these sensors are bringing in. The work that we did in the custom silicon that we designed does a lot of what the XR1…these chips, are doing.

We improved that HPU in the HoloLens 2 with the addition of embedded deep neural networks in the silicon. Back in July of 2017, I think at the Computer Vision Conference in Hawaii, Harry Shum [executive vice president of Microsoft's AI + Research Group] demonstrated some of the HPU 2.0's capabilities. At the time, we talked about fully articulated hand tracking, but nobody really picked up that, "Hey, that's gonna be a feature of the next HoloLens."

So we've invested a lot in the coprocessor and the technology to enable these mixed reality experiences, and we really like the work that the team has done, and we're building on it. At some point, it may become a more commoditized thing, where we don't have to do our own special design, but we have a lot of IP invested in it, so it's actually a differentiator for us.

Next Reality: Regarding the design of the HoloLens 2, one of the early, unofficial images I managed to get a look at of the device before its release included a kind of Oculus Rift-style top-of-head strap included in the design. I didn't see that anywhere in the official HoloLens 2 presentation. Was what I saw just some sort of early prototype component?

Sullivan: In the same way that the existing device [original HoloLens] has a strap that you don't always see, but it comes in the box, this one [HoloLens 2] has a strap. In the first one, [the strap] could potentially have been more of a comfort thing, but I think it was more of a stability thing to keep it kind of in place. This one [HoloLens 2], you'll see, it fits on your head like a baseball cap, the center of gravity is further back, it's much more balanced. And it's a little bit lighter, 13 grams lighter than the original HoloLens, but it feels much, much lighter because it's balanced on your head.

Next Reality: I've written in the past about my concerns around the idea of putting immersive computing devices with large, heat emitting components and battery packs on your head. With the HoloLens 2, everything, including the battery, sits at that back of the head. Have you done any kind of extensive testing to see if there are any issues with regard to heat and comfort?

Sullivan: The HoloLens 2 is fanless, so thermal management is a priority that the team spent a lot of time engineering around. We have this new, innovative vapor chamber that serves as a heat sink to promote that fanless thermal management. So that's always a consideration, but there's also a lot of compute back there, it's not just battery. In the first HoloLens, it was all sensors and all compute up front, and batteries on the sides, and nothing really on the back.

Next Reality: So now on the side headband areas, it's "just" spatial audio components, one on each side?

Sullivan: Yes, and the buses, the data buses to connect the front to the back.

Next Reality: Aside from the improved hand tracking, lighter weight, and a few other things mentioned in the initial reveal, explain why you think the HoloLens 2 is so much better than the first.

Sullivan: There's really three areas that we focused on, and it there based upon what the existing HoloLens customers told us they loved about HoloLens, [what] they wanted to see [made] even better. The three top priority areas were immersion, comfort, and what we call out-of-box value. Immersion is largely equated to field of view. It's more than that, but the number one characteristic that improves your sense of immersion is how big the holographic frame is. So [HoloLens 2] is more than double the area.

Next Reality: So there's been some controversy over this regarding the claim in the initial stage presentation that the HoloLens 2 has twice the field of view compared to the original HoloLens. I think Alex Kipman weighed in on Twitter.

Sullivan: The controversy was, when you say you've doubled the field of view, are you talking about the horizontal degrees of field of view or are you talking about the area, and our response is that we're talking about the area. Because we actually did increase the horizontal — we went from 34 degrees to 52 degrees on the horizontal — but we increased even more in the vertical because the vertical constriction is what kind of made it feel like a letterbox view, where [a virtual characters/objects] disappeared. The first Hololens used a 16:9 aspect ratio and now it's 3:2. So we dramatically improved the vertical field of view and that has led to a greater sense of immersion.

Next Reality: In terms of immersion, there's also the new Azure Kinect sensor…

Sullivan: Yes, one of the benefits we get from this new HPU coprocessor is that fully articulated hand tracking, combined with the improved depth sensor that we have. That enables the fully articulated hand tracking so that we can get direct manipulation of holograms.

You remember, with the first HoloLens, you had to do a very specific gesture, and it's kind of an analog to clicking a mouse, with your gaze being the mouse pointer, and [the hand gesture] being the click of the mouse.

Well now, we have not only the ability to just reach out and grab a hologram and rotate it and resize it and move it, without using any specific gesture, just naturally in whatever way you want to grab it, we added to that eye tracking, so that instead of having to have your head pointed at something to have that [mouse pointer] dot show up, you can just look at stuff and make things happen. It makes you feel closer to the digital content, less abstracted with a weird gesture.

• Don't Miss: Microsoft's HoloLens 2 Is an Enterprise Workhorse, but It's Sexy, Too, Here's a Close-Up Look

Next Reality: Honestly, that [original HoloLens pinch] gesture always bothered me. It was always one of my pet peeves about using the device. It was ok for the time, but I'm glad we're here now.

Sullivan: The next area was comfort. Customers said this is great, but we'd love to have some of our people wear this for hours and hours at a time without neck fatigue. And it needs to fit a wider variety of human heads without being uncomfortable. So the team did a whole bunch of work modifying the device so that it would fit a much wider range of the physiological variability of human heads.

Next Reality: It definitely feels better. And I think the flip-up visor addition is particularly helpful in terms of making extended use of the device easier.

Sullivan: The third thing was out-of-the-box value. With the original Hololens, years ago, it didn't do a lot out of the box. You had to hire software developers and spend three to six months building an application, which was time consuming and costly. One of the other feedbacks that we got from customers was, "I want to be able to start doing things right away without doing a bunch of custom app development work."

That's part of why we had this mixed reality suite of applications from Lorraine Bardeen's [general manager of engineering for the Dynamics 365 mixed reality apps group at Microsoft] team where we have Dynamics 365 Remote Assist, Layout, and now Guides. (*See more about Bardeen and her perspective on augmented reality in the video below.) So think of it as the Word, Excel, and Powerpoint for HoloLens, a suite of applications right out of the box you can get value from.

Next Reality: I think I understand why the focus with HoloLens 2 is on enterprise. However, after seeing so many cool executions on stage (at one point, a user played a virtual piano), and considering some of the non-enterprise applications your competitors (read: Magic Leap) are focused on in the way of gaming, as well as enterprise, I'm wondering why you're "just" talking about enterprise.

Sullivan: It's very clear, HoloLens 2 is a commercial enterprise device. And we're very focused on that market segment with this generation because of the value that's derived from enterprise customers. We're gonna have our hands full just meeting the needs of enterprise customers.

• Don't Miss: How to Pre-Order the HoloLens 2

I also think it's not unlike many technologies that have come into our worlds and into the consumer space. They actually were adopted in the commercial sector and then achieved critical mass and scale there, which then drove economies of scale to drive the cost down, then they could become consumer [products]. We've seen that with the original PC, with cell phones, with microwave ovens, and radio.

There's a whole list of technologies that have been adopted in enterprises because the ROI is so impactful and profound and they pay for themselves. It's a lot harder to justify that $3,500 device if it's [just] for me to play games on it. But if it saves my business a lot of money, it's a no-brainer.

• Follow Next Reality on Facebook, Twitter, Instagram, YouTube, and Flipboard
• Sign up for Next Reality's daily, weekly, or monthly newsletters
• Follow WonderHowTo on Facebook, Twitter, Pinterest, and Flipboard