Facebook Reality Labs, the company’s R&D department, previously revealed its ‘Half Dome’ prototype headsets which demonstrated functional varifocal optics small enough for a consumer VR headset. At a conference earlier this year, the Lab’s Director of Display Systems Research said the latest system is “almost ready for primetime,” and also detailed the Lab’s research into HDR (high-dynamic range) and pupil-steering displays for XR headsets.

Technological Readiness

Douglas Lanman, Director of Display Systems Research at Facebook Reality Labs, gave a keynote presentation at the SPIE AR VR MR 2020 conference earlier this year. In his presentation, which was recently posted online, Lanman introduced a scale of ‘technological readiness’:

  1. Basic Research – Basic principles observed
  2. Technology Formulation – Technology concept and application formulated
  3. Initial Validation – Experimental proof of concept
  4. Small Scale Prototype – Technology validated in lab
  5. Large Scale Prototype – Technology initially validated in intended environment
  6. Prototype System – Technology robustly demonstrated in intended environment
  7. Demonstration System – System prototype demonstrated in operational environment
  8. First of a Kind Commercial System – System complete and qualified
  9. Generally Available Commercial System – Actual system proven in operational environment

While the scale was originally used by NASA, Lanman likened it to the journey that research (level one) takes all the way through widespread availability of a product (level nine).

Lanman explained that the work of researchers tends to focus on levels 2 through 4, at which point the research is published and the researchers move onto another project, but rarely see their work reach the higher levels on the scale.

The Display Systems Research team at Facebook Reality Labs is unique, Lanman said, because the group has the capacity to work between levels 1 to 6, taking research all the way from “first principles” through to polished prototypes; much closer to a finished product than researchers typically see their work carried.

“So what’s really unique about this Display Systems Research team is that we’re not quite a startup, we’re not quite a major company, and we’re not quite academics. We really play from the absolute fundamental vision science through very polished prototypes—more polished than you’d see from most startups—to try to do one thing, which is [have a genuine impact on future products].”

Half Dome “almost ready for primetime”

Image courtesy Oculus

The team created a series of prototypes dubbed ‘Half Dome’, which employ varifocal displays that allow the headset to correctly support both vergence and accommodation together—something no consumer VR headset does to date.

Half Dome 3 is the latest of the prototypes which Facebook Reality Labs has spoken about publicly. Instead of relying on a mechanically-driven varifocal display like the prior prototypes, Half Dome 3 implemented a static varifocal display which uses a series of liquid crystal lenses that allow the headset’s optics to change between 64 discrete focal planes. Half Dome 3 also employs ‘folded optics’ which significantly reduce the size of the display module.

Size comparison: Half Dome 3 static varifocal display module with folded optics (left), Half Dome mechanical varifocal display module (right) | Image courtesy Oculus

The first Half Dome prototype was revealed back in 2018. At the time, Oculus said that customers shouldn’t “expect to see these technologies in a product anytime soon.” A year later the Half Dome 3 prototype was revealed, but Oculus was still tight lipped about whether or not the tech would find its way into a headset.

While it’s still not clear how close Oculus is to productizing a varifocal or folded optic display, Lanman ranked both Half Dome and Half Dome 3 on the technology readiness scale that he introduced earlier in this talk.

He placed Half Dome, the mechanically-driven varifocal headset, at level 6 (Prototype System), and Half Dome 3, the static varifocal headset with folded optics, at level 5 (Large Scale Prototype). “It’s almost ready for primetime,” he said of Half Dome 3.

Continue on Page 2: A Step Toward “creating the world’s first HDR headset” »

1
2
Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Till Eulenspiegel

    The team build a benchtop prototype to prove out the pipeline and created a working HDR display capable of 6,000 nits of brightness (compared to around 100 nits for the typical VR headset, according to Lanman).

    That’s 60X brighter, you’ll need to wear sunglasses for this headset.

    • Marian

      Yes, but usually the higher brightness levels are used only for specular highlights, smaller areas. Like with HDR BluRay movies. The typical scene won’t be too bright for the eyes.

    • dk

      no u wouldn’t but 100 sounds low

    • kontis

      You just cover your “eyes” with your tracked hand.

    • Ad

      Pressed right up to your eyes. And anyone who thinks that’s not an issue has clearly never had a program crash on them and seen random colors, white screens, etc.

    • mfx

      that is because they use tons of polarizing filters that they need mega bright screens.

    • Well that’s really 6,000 divided by, up to, 64 possible layers. There’s a ton of possible loss in that system. They would likely turn down the brightness based on how much of that 6,000 possible is being blocked at a time. It probably evens out to a normal 100.

      But as another person here mentioned, there is always the possibility of the brightness being stuck on full with all LCD layers turned off.

    • Dave

      LOL it’s not 6,000 nits all the time, just when you look at bright objects er like the sun. Sony already has tech which can achieve 20,000 nits.

  • kontis

    He said one thing that is not actually true.

    To date, no one has tried HDR in VR..

    Valve had a HDR prototype headset years ago

    We don’t even know if it matters.

    Yes we do. It was apparently amazing and everyone was impressed.

    The thing we want, of course, is brighter brights. I really do want to go to a beach in VR and be blinded by the sun.

    And the test scene they used was literally that – a beach in VR with blinding sun

    Nice research, but I recommend googling next time before making a presentations with claims of “never done before” experiments.

    And even if Valve didn’t use high contrast lenses for that – it still counts. The difference was extremely noticeable and positive, so it worked.
    Valve is once again years ahead of Facebook.

    • benz145

      I would guess the Douglas is well aware of the experimentation around HDR in headsets; I took his statement to mean that “no one has shipped an HDR headset yet.”

    • DeenVR

      > Valve had a HDR prototype headset years ago

      Proof?

      • Greg

        ABSOLUTEMENT NON!
        YOU WIN THE ARGUMENT
        NIC CAGE’S HAIR IS BIRDLIKE

    • Andrew Jakobs

      Nah, don’t really need brighter brights, I want lenses that don’t have the godawful godrays, and a very large sweetspot.

      • polysix

        Reverb G2 (minimal god rays.. ALL sweetspot – best clarity). LCD (but good LCD) blacks but other than that…

    • Dave

      HDR in a headset is going to be super amazing and super challenging. I mean if you’ve bought one of Asus’ HDR beasts with local dimming you’ll know the high power consumption and heat it generates is insane!

  • Ad

    Considering they’re only interested in making sub $400 standalone devices, I seriously doubt it.

  • Rudl Za Vedno

    Facebook Reality Labs Says Varifocal Optics Are “almost ready for primetime,” BUT it will cost FB mucho $$$ to produce. Zuck decides:” Nah, dirt cheap Quest S will do just fine. Our VR users are used to shitty experience by now.”

    • ViRGiN

      Something tells me you own a pimax and think you’re on the highest end of vr.

      • Rudl Za Vedno

        No, I own HP Reverb and G2 is on the way.

        • Kataki kun

          You’re just a fucking spoiled brat

          • Yup — I can smell entitled rich kids a mile away.

          • John

            Some can smell salty peasants a mile away too… lol.

          • WOW — did you really just call me a peasant, you Bourgeoisie piece of shit?

          • John

            The poor often get angry at the successful and the wealthy because they’re angry about being failures… lol.

          • Damn — and the rich just prove what USELESS GAPING ASSHOLES they are.

    • So don’t buy an Oculus. Buy an Index if that’s your issue. Some of us WANT a standalone device under $400.

      • R3ST4RT

        Yup, I own an index and a quest, and the quest gets waaaay more usage because of the wire-free experience. The index is by far the better visual experience but the quest just makes things convenient.

        • John

          The Valve Index can have an essentially wireless experience as well using a low-cost pulley system for cable management, as shown here:
          https://www.youtube.com/watch?v=H53QmxZ-rOs&

          • That’s a stupid-ass statement. I use the same pulley system on my Quest when connected to my PC and there is no way in HELL you can call it “an essentially wireless experience”.

        • StaticNebula26

          Yes the quest is convenient but imagine if it was built to index specs, you wouldn’t have to choose between them. that’s why some, myself included, want a quest pro like headset, even if it’s (hopefully) not made by Facebook. The quest will have it’s place of course but the standalone idea is being very underused.

  • wheeler

    It’s hard to gauge exactly what “almost ready” means when Abrash keeps pushing out the expected date for a substantial upgrade–and also what “prime time” means (just an awesome internal prototype? or a commercially viable product?). But 2024 (if we’re going by Abrash’s latest projections) wouldn’t actually be that far off in research terms. Have they addressed the issues with eyetracking accuracy that are holding this and other technologies back?

    I also worry that FB’s desire to hit the $400 price point may delay this further–what does a varifocal system like this cost? They’re clearly willing to cut down on audio, IPD adjustment, ergonomics, computational power, and refreshrate to hit $400. You can have the most amazing researchers in the world but if your business model prevents you from actually utilizing the results of that R&D, then what’s the point?

    Nonetheless I’m really anticipating the day that I can get my hands on a varifocal headset. I don’t see VR taking off until a variety of hardware problems are solved and this is one of them.

    • Bob

      “But 2024 (if we’re going by Abrash’s latest projections)”

      Sorry what?

      • wheeler

        https://uploadvr.com/oc6-next-gen-not-soon/

        Correction: it’s 2023 or later

      • Dave

        @disqus_4e2PIma4qr:disqus LOL you’re not a fan of Oculus Connect haha! This is pretty well known Micheal Abrash makes predictions in every year.

    • Rogue Transfer

      Michael Abrash has most recently spoken last month, saying about eye-tracking: “As a result, it still remains to be proven that it’s possible to track the eye accurately and robustly enough” To hear his talk, go to Youtube dot com and add a slash and: watch?v=NNWUnn-lW9g&t=790

      So, it’s not “almost ready”, it’s not even proven possible.

    • mfx

      yeah make a 300 model for masses and a 700 one with the best tech for nerds

    • Yah, seems like another project Facebook is doing just for the patents.

      There is no drive in their ranks to chasing the cutting edge. Carmack seemed like the last real driving force at Oculus, and after the Quest’s completion, he just stepped down. With no leadership left, it’s just dangling in the winds of a soulless parent company only interested in picking the flesh from it’s bones.

      Worst case scenario, Facebook uses this patents to squeeze other VR companies and we NEVER see these innovations in a consumer product. Man, I hope not… but odds favor it.

      • Bob

        Michael Abrash is still the Chief Scientist at FRL.

    • mirak

      They will probably target professionals.
      I think varifocal helps reading even more than resolution.

  • M

    I’m curious to know what the goodies are on the back of the headset. Looks like some kind of sensor on the back of the head, plus a little place to clip in a processor?

  • impurekind

    Good stuff.

  • brubble

    meh.

  • Those folded/holographic optics still blow my mind. cannot wait to see one or both in a headset.

    • Rogue Transfer

      They suffer from laser bounce speckling, ghosting and blur – according to Facebook Reality Labs latest research PDF on it, they published online recently.

      They even included some images and a short clip showing some through-the-lens in an accompanying article.

  • “almost ready” so, 2025 at the earliest.

    • dk

      nah 2022 ….which is not really that much of a difference :P

  • Sven Viking
    • Thanks that’s a great link. Loving this quote:

      “Valve created a HDR prototype headset in their lab: “The worst part is, every time from then on that you’d ever use a normal headset you just feel really sad and disappointed”.

      • Sven Viking

        It might be too hot or even potentially dangerous to the eye at that level, but if so there must be something in-between that would be practical but still much better than what we have now.

        • Index at 100% brightness (it now goes to 160%) is only 95 nits, unsure what it is at 160% but it still seems relatively dim compared to Vive.

          To give some figures, quoted by Redditor “Eagleshadow” who did direct testing on these headsets:

          Luminance in nits:

          Index: 95
          Vive Pro: 143
          Vive: 214

  • 6000 nits???
    I’d prefer not to go instantly blind kind sir.

    • StaticNebula26

      I mean the point is, I assume, to make suns more realistic by making it hurt to look at them

  • duck

    Exciting times ahead.
    Thumbs up

  • I think he wanted to raise some hype. Level 5 is still a prototype, a good prototype, but still a prototype, and going from prototype to product is very hard. If “Almost ready” means 2-3 years, then ok, it is almost ready

    • benz145

      I think his point was that the technology is ‘almost ready’ to be commercialized. As in, they got it good enough in the lab setting to show that it could be a viable technology in a product. But that’s still a far step from having it actually integrated into a real product (have to figure out firmware, rendering, manufacturing and all of that).

      • Bob

        “But that’s still a far step from having it actually integrated into a real product”

        If they’re moving forward with Half Dome 3 then I would assume no more than two years and it will be ready for mass production by 2022.

  • Since we’ve never seen any form of variable focus, outside of the two layers the Magic Leap supports, it’s a bit hard to grasp what these 64 layers will actually mean to the end user. Images already seem in and out of focus with simple stereographic vision. I’m certain it would be a good thing… but would it be as useful as actual Foveated Rendering with the use of eye tracking? Would it be as much of a game changer as a larger field of vision, or retina-quality resolution? Or the massive reduction in size that the LYNX MR headset is promising?

    It seems like they are working really hard to innovate on something that will, as far as I can tell, be a minor upgrade to the optics. The Holographic Folded Optics project seems far more promising.

    • dk

      it’s not a minor thing …it’s huge …it’s making close up things in vr not look blurry ….it’s just as important as foveated rendering saving gpu power and high angular resolution

      vr displays have a fixed focal length somewhere around 2-3 meters ….that’s why when u look at your controllers close us they look blurry because your eye wants to focus at 20cm while the image is at 2.5 meters ….eye tracking and foveated rendering can’t help u with that objects close up will always be blurry if u don’t have varifocal or light field display …unless u have presbyopia then close up things in vr look sharp without your glasses and u need glasses for the real world with close up objects

      they r working on everything at the same time ….and the Holographic Folded Optics is just one of the things which if there r no problems might be promising in 5-10 years

    • benz145

      The biggest benefit is fixing the vergence accommodation issue. These varifocal optics are closer to replicating how we see the real world than the simple optics we use today. It will ‘feel’ more realistic, even if it’s difficult to quantify, and it’ll be important for long term use.

  • FMT

    IMHO later this year Oculus will release new Quest with 20% more FoV and XR2 chip, based on larger lenses we see on leaked pictures and cheaper look to balance the expensive chip.
    Spring next year they will release new Rift with varifocal and eye tracking, for now it can only be done on Rift because of high power demand to operate very bright screens.

  • Moe Curley

    I believe that using this type of varifocal will preclude wide FOV so I think we’ll presented with a choice soon. I for one would favor a high res wide FOV headset over varifocal. I want out of the diving mask badly.

  • JS

    How about resolution? Without a high pixel per degree ratio VR won’t be ready for things like virtual movie theaters.

  • Paul Acosta

    I imagine this technology will be available in 2-3 years at best and I don’t think this hype makes any sense yet.

  • polysix

    LMAO – Facebook trying to spoil the glorious Reverb G2 pre-sales with leaked ‘news’… I’m sure there will be a great ‘rift 2’ at some point from Oculus but the timing is very obvious.

    • mfx

      These infos were extracted from a professionals to professionals talk, and even if it was true, they are doing some good work to push the RnD to the next level, so there is nothing to complain about.
      Don’t worry if the reverb g2 is really good, it will sell, and it’s not a few articles about RnD, read by the 1 percent most vr enthusiasts, that will waste the success of it..

  • Michael Lupton

    Half Dome 3 confirmed

  • johnvvr

    I agree, it’s usually not the best VR way to do that. But who knows! We’ll just have to wait and see I guess :)