Varjo is a relatively new name to the VR scene, but the company is certainly making a buzz, having quickly raised some $15 million in venture capital touting the promise of delivering a VR headset achieving retina resolution at the center of the field of view. But how exactly does it work? A new graphic shows the key tech behind the company’s headset.

Today there’s primarily two choices when it comes to the kind of displays to use in VR headsets. The first is a traditional display like the kind that you find in your smartphone. The problem with traditional displays today is that the pixels aren’t yet small enough to be truly invisible. Then there’s microdisplays, which have incredible pixel density, but the displays themselves can’t yet be made large enough for very wide field of view in a VR headset.

So until either traditional displays can shrink their pixels drastically, or microdisplays can be easily made much larger, we’re still quite a way from achieving ‘retina resolution’—having such tiny pixels that they’re invisible to the naked eye—in highly immersive VR headsets.

But Varjo hopes to deliver a stopgap which combines the advantages of traditional displays (wide field of view) with those of microdisplays (high pixel density), to deliver a VR headset with retina resolution (at least in a small portion of the overall field of view).

Combining Displays

The underlying concept is illuminated in a recent animated graphic from the company’s site:

Above you can see (right to left) a diagram of the viewer’s eye, a traditional lens, a moving refraction optic (above), a microdisplay (below), and a traditional display.

As you can see, the refraction optic can move the reflected microdisplay image onto the corresponding section of the traditional display. The idea is that the reflected high-resolution image will always be positioned at the very center of the user’s gaze, with the help of precision eye tracking, while the lower resolution traditional display will fill out the peripheral view where your eye can’t see nearly as much detail. This is very similar to software foveated rendering, except in this case it’s almost like moving the pixels themselves to where they are needed, instead of just rendering in higher quality in a specific area.

This example image from Varjo shows the difference in visual quality between the image from the traditional display and the microdisplay (click to zoom):

Image courtesy Varjo

Moving the High-res Region

The big questions of course: how do you move the refraction optic quickly enough to keep up with the movements of the eye, reliably, and in a space compact enough for a reasonably sized VR headset?

Image courtesy Varjo

For the former, the answer may lie somewhere in the company’s key patent, Display Apparatus and Method of Displaying Using Focus and Context Displays, which describes “actuators” that could be involved in the various moving parts. Additional hints pertaining to how the company hopes to achieve this are likely found in Varjo’s job listing for a “Miniature Mechatronics Expert:”

You will be responsible of designing the actuators and motor controls for our mixed reality device. […] You will participate the development of leading edge motor technologies and design novel actuator mechanics to harness the power of custom designed optics, motors and electronics to reach new fronts in miniature mechatronics.

[…]

Responsibilities

  • Create motor position control algorithms
  • Design position encoder system
  • Set performance targets and requirements for motor units
  • […]

Plans ‘A’ Through ‘H’

The patent, which refers to the traditional display as the “context display” and the microdisplay as the “focus display,” actually covers a wide range of possible incarnations of the tech using varying methods for combining the microdisplay image with the traditional display image, including the use of waveguides, additional prisms, and other display technology entirely, like projection.

Image courtesy Varjo

Correcting Artifacts

Another big question: what artifacts will the optical combination process introduce to both the reflected micro display image and the image from the traditional display?

The patent also touches on that (a clear indicator that artifacts will indeed need to be contended with); it suggests a number of techniques which could help to eliminate distortions, including masking the region of the traditional display that’s directly behind the reflected display, dimming the seams of the two display regions to try to create a more even transition from one to the other, and even intelligently fitting the transition seams to portions of the rendered scene, connecting them like puzzle pieces in an effort to make the seams less pronounced.

– – — – –

Image courtesy Varjo

Varjo has laid out a number of interesting methods for pulling off their “bionic display.” Of course, the devil is always in the details—we’ll be looking forward to our first chance to try their latest prototype and see what it really looks like in practice.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Jistuce

    So THAT’S how they’re making the high-res inlay work! Clever, very clever.

    • Kyra

      Gℴℴgle giving to people of every age 98 US dollars hourly to complete some work with a computer .. Work Some just few hours daily & have more time together with your own friends … Anyone can benefit this best post…last Monday I purchased a brand new Saab 99 Turbo after just earning $12458 this five weeks .it looks the easiest-work however you will no longer forgive yourself if you don’t take a look at it.!sx86o:⇆⇆⇆ http://GoogleCashCareerNewWorkFromHome/find/jobs ♥♥♥q♥♥n♥♥♥u♥o♥l♥♥♥z♥♥h♥j♥u♥♥♥h♥f♥♥k♥♥n♥♥♥a♥r♥x♥♥♥u♥♥♥x♥r♥f♥o♥s♥y♥♥♥y♥♥♥w:::::!ew02u:chgwj

  • dk

    sooo the question is …..when can people try it ….and when will someone produce it

    • MrGreen72

      Well they have to find that Miniature Mechatronic Expert first so he can solve the remaining pieces of the puzzle right?

      I’m no expert but this sounds over-engineered to me and I’m not sure that having high precision moving parts in an HMD is a good idea.

      Why can’t microdisplays be made larger? Or stich a few together? Aren’t they bezel-less already?

      I have the feeling this will never reach the market, but I sure wouldn’t mind being wrong…

      • benz145

        They have a tech demo/prototype; they likely have enough in-house knowledge to get them to a minimum viable prototype, but now that they’ve raised funding they’re bringing in experts in various fields to design it all from the ground up—pretty typical startup approach.

        It also took me a long time to understand why microdisplays can’t easily be made larger; after asking people who work in the field, I came to learn that they are made using the same fabrication processes as computer chips, which requires billions of dollars in equipment and facilities to achieve. And since those fabrication processes are built with the goal of making the chips as small as possible, they can’t be easily rejigged to make them bigger, without investing a ton more money.

        It is on the roadmap for microdisplay makers like Kopin to make bigger displays, but it’s going to take them a few years to get there. Lots more detail here:

        https://www.roadtovr.com/kopin-prototype-vr-headset-lightning-microdisplay/

        • Michael

          So essentially we have the technology to make ultra-high-res displays, this isn’t something in the future but something here today… but that the displays are extremely expensive to make due to having to rig entire microchip line for it.

          As long as I know we’re capable of getting these huge but extremely dense displays then I’m happy. Maybe the Rift 2 will have something close… but I think it won’t be a microdisplay at all, but rather a 4K display but condensed.

        • Darshan

          Very Eagerly waiting for read review of VRJO working prototype here… and i mean it very eagerly.

          • Annette

            Gℴogle giving to people of every age $99 every hour to complete easy jobs from the comfort of home .. Labor Some only few hours daily & stay more time with your relatives … any individual can also avail this super post…on Sunday I purchased a great Fiat Panda after just getting $17520 this last five weeks .without any doubt it is the best work but you will not forgive yourself if you do not have a peek at this.!wf683u:➻➻➻ http://GoogleNetJobsStarCashJobsOpportunity/simple/work ♥♥y♥♥♥u♥p♥♥♥p♥l♥i♥♥h♥y♥♥♥a♥♥♥w♥♥a♥♥♥a♥v♥r♥♥♥h♥♥g♥g♥♥w♥♥q♥♥♥n♥u♥♥e♥a♥u♥♥♥t:::::!xg401z:wks

      • dk

        they have a prototype …and further funding to develop it

        it’s made this way because within a year u could have a headset with resolution close to the 20/20 human vision …………..aaaand waiting for a traditional headset display to get there which will be like 16k by 16k per eye will take more than 10 years

  • NooYawker

    I’m excited for this. The screen door effect is a huge barrier for VR.

    • MosBen

      I disagree. Don’t get me wrong, I will welcome the day when we don’t have it, but when I have non-enthusiasts try my Rift, few to none have complaints that can be traced to the screen door effect. Cord management and the ergonomics of the HMD are easily the biggest issue that I hear. Behind that, tracking issues, particularly with the controllers, are a significant factor. Behind that, the porthole effect of the current field of view comes up. Of course, the fact that I’m the only one of my friends that has a VR headset probably implies that more than any of these issues, cost and difficulty of setup are probably the biggest barrier of entry.

      Screen resolution is probably one of the easier problems to fix, so it will probably be addressed relatively soon. But my impression is that relatively few people are holding off on getting into VR until there’s no more screen door.

      • NooYawker

        Well luckily different companies are working on different things. Minimum specs is a major hurdle. The average person doens’t have a powerful gaming rig. Price, weight, cordless.. these are all things affecting VR. Also I don’t think resolving screen door effect will be that easy. To get each pixel down to a point that it’s not noticeable that close to the eye is going to take a lot of research. Little nanopixels or something new.

        • Darshan

          Arent we just months away from OCULUS GO , all in one that does not need pc, is cordless, doesnt need phone to be slot in or adjusted, improved optics yet just $200 entry price..

          Minus point is no 6DOF controller support and no positional tracking but same can be worked out in next versions.

          Providing people something with 50+ Apps at start with just $200 price tag and console like platfrom for developer to target is nice possiblity for VR FUTURE

      • Ben

        To each his own, and I guess it depends on the kind of apps you prefer.
        They may not be a barrier to entry, per se, but SDE, FOV and lack of resolution are the frustrating reasons I hardly use my rift anymore. That, and VAC, which is very tiring for my eyes. I don’t feel any of the other reasons you cite as much a barrier to « presence ».
        Not being able to read button labels, a paper map or a GPS screen in a flight simulator is very frustrating. Same for wanting to see more in the distance in Google Earth, or to experience a movie as in the theater, in full res.

        • Guest

          Obviously, bi-anything = vergence accommodation.

    • nipple_pinchy

      I don’t even notice SDE on my Odyssey. There’s much greater hurdles for VR to tackle like going wireless, being smaller and lighter and especially becoming cheaper.

    • brandon9271

      The 1# offender for me are the damn out of focus, smudgy, god ray inducing fresnel lenses! Who cares about pixel density when the pixels we DO have look like crap? SDE effect is the last thing I’m concerned about when it VR. It’s trying to get the HMD in the EXACT sweet spot and hope to go it doesn’t move and try not to play any high contrast games that make me feel like I’m playing behind goggles smeared with vasoline. I’m hopeful that Valve’s new custom lenses solve these issues.

  • It is similar to what I thought when I read about their idea… the problem is: will they be able to produce something like that?

  • Yoan Conet

    What about shrinking pixel of a non a physical screen but pixel of a video projection into a “point” to get cheaply a huge pixel density video projection…
    No need of expensive microdisplay then hihi

  • Robert1592

    The fact that they’re still looking to find a Miniature Mechatronic Expert to work on this issue is a bad sign. They think it can work but they don’t know for sure because they don’t have anyone sufficiently expert in such things to tell them if they’re being realistic or not. Jeez if I was investing 15 million dollars in a company I’d want to see that Miniature Mechatronic Expert already on staff and telling me it will work and already showing off the prototype.

    • El Pingüno

      probably that’s why you aren’t vc investor?

      • Robert1592

        I’m not a VC investor because I don’t have the cash to waste on people who don’t know what they are doing. If I was a VC investor I’d expect more proof that they could do what they say rather than just show me a drawing of an idea and give me a promise that they’ll try real hard.
        There seems to be a wave of gullibility passing through the world right now.

        • El Pingüno

          that’s why you don’t have a cash in the first place. These investors, apparently, ready to take this risk and fund a promising team with idea.

          • Robert1592

            The idea is not promising. There’s some serious technical problems. Problems that I think will not be solved by them. People who invest in dumb ideas don’t have money for very long. I have cash but not to waste on dumb ideas.

          • El Pingüno

            like renting our peoples houses for couple of days? maybe build electro cars when there are no any infrastructure for that? Personal computer seemed like dumb idea for very smart people back when. Even if 99% VC’s risky investment fails, 1 will succeed on the empty market, because everyone else thought that “The idea is not promising. There’s some serious technical problems. Problems that I think will not be solved by them. “. If idea is promising and tech seems implementable there would be tons of competitors. Simple as that.

        • Ref caller

          How do you know they didn’t vet them already? You don’t stay in VC long if you are bad at it. Typical armchair refereeing.

          • Robert1592

            You can’t really VET something that hasn’t been invented. You’re basically taking the word of the CEO that they can invent it. You need to have a decent level of expertise and a mental defense against hype to judge it. VC types tend to be money people and the sheer volume of money invested into this thing suggests unreasonable expectations. Of coarse if you have some detailed information as to what has gone on here I’d love to hear it. It would make interesting reading I expect.

    • brubble

      +1. A bunch of design idea/concepts scribbled on napkins does not inspire confidence. Gotta start somewhere I suppose.

  • Peter Hansen

    What I don’t understand: If they have eye tracking fast enough to make such a headset (if…) – and keeping in mind that the GPU/driver tech for foveated rendering already existes – why the HECK go for a mechanical solution? Just wait another year, and the high-res displays are there. Together with well-enough data compression (for the PC interface) we don’t have to re-invent the wheel.

    Think about the additional cost for parts, or the very high needed precision of such actuators. Imagine to drop you headset to the ground… how resilient those mechanics have to be.

    Am I the only one who asks “why”?

    • El Pingüno

      well, if you drop vive right now – don’t think anything good is going to happen. But overall yes, seems like logical solution to wait but the investors wanted to hedge risks

  • j drew

    dont DLP projectors kind of work like this? instead of a screen why not use a form of DLP projector along with eye tracking? sounds a bit sci-fi i know

    • The nano-machine mirrors in DLP only flick on and off, so mechanically, they only have to accelerate between two points. There are clear stops in the structure to prevent it from moving further. This will require a much larger single mirror to move smoothly without hard stops. Even if it’s weight it small, the speed involved will still carry alot of inertia. I’ve been toying around with some small actuators in a similar size. Once you get that small, precision becomes hard and things tend to shake.