This week Epic Games released the latest version of its next-gen game engine, Unreal Engine 5. Though the new version brings improvements in many areas, its most notable features are Lumen (global illumination) and Nanite (micro-polygon geometry), which could be game-changers for VR immersion. Unfortunately the company says neither feature is ready for VR developers.

Available as of this week for all developers, Unreal Engine 5 promises to usher in a new era of game development which makes it easier for developers to create games with extremely high quality assets and realistic lighting. That’s thanks to the engine’s two new key features, Nanite and Lumen.

Nanite

Nanite is what Epic calls a “virtualized geometry” system which radically improves the geometric detail in game scenes.

A rea-time scene rendered using Nanite | Image courtesy Epic Games

Previously developers would create high quality 3D models as a sort of ‘master’ reference which would eventually have their geometry greatly simplified (leading to a reduction detail and complexity) before being pulled into the game engine. The same model generally gets several versions with increasingly reduced detail which ‘pop’ between each other depending upon how far the game camera is away from the model (known as ‘level of detail’ or ‘LOD’). This allows the game to show higher quality up close while using the reduced quality models when they are further away to save performance.

Nanite essentially functions like a continuous LOD system that draws detail from the original ‘master’ model, instead of relying on pre-built models with reduced detail. In each frame the system references the master model and pulls out the maximum level of detail needed for the given camera distance. Not only does this eliminate the need to create discrete LOD models, it also means that the range of detail for a model can be much greater, allowing players to see incredibly fine detail—right down to the original polygons of the ‘master’ model—if they get close enough.

Lumen

Meanwhile, Unreal Engine 5’s new lighting system, Lumen, greatly simplifies game lighting thanks to global illumination.

Real-time lighting rendered with Lumen and Nanite | Image courtesy Epic Games

Realistic lighting can be very computationally expensive; without Lumen, many games use a combination of lighting techniques to achieve the look they want while maintaining game performance. A given scene might use pre-calculated ‘baked’ lighting (which isn’t interactive with the rest of the scene) along with a small number of real-time light-sources that cast shadows on certain objects in the scene, and various ‘screen-space’ effects to emulate realistic lighting.

Lumen unifies lighting into a single approach called global illumination, which aims to make every light in the scene—even the Sun—into a real-time light that is interactive with other lights and the rest of the scene. This includes realistic bounced light which spreads color throughout the scene based on the color of the objects that the light hits. For instance, white sunlight shining into a white room with a red floor will cast some red light onto the walls as it bounces from the red floor. Such bounced lighting is an essential component of photo-real lighting.

Both Nanite and Lumen could massively improve immersion in VR thanks to their ability to hugely increase geometric detail in nearby objects (which is especially noticeable with the stereoscopic capability of VR headsets) and to create more realistic and interactive real-time lighting.

“No Timeframe” for Nanite or Lumen in VR

Unfortunately Epic says that neither Nanite nor Lumen in UE5 are ready for VR yet.

“While we have no timeframe to share in terms of Lumen and Nanite support for VR experiences, we are exploring how to bring those UE5 features to additional platforms,” the company tells Road to VR.

But, Epic says, that doesn’t mean VR developers shouldn’t use UE5.

“VR developers can leverage most of Unreal Engine 5’s production-ready tools and features, such as the new UI, the new suite of modeling tools, creator tools such as Control Rig, MetaSounds, and World Partition for large open environments.”

What’s the Holdup?

Though both Nanite and Lumen are capable of creating incredible looking scenes, they aren’t ‘free’ from a performance standpoint.

“Although the advantages [of Nanite] can be game-changing, there are practical limits that still remain. For example, instance counts, triangles per mesh, material complexity, output resolution, and performance should be carefully measured for any combination of content and hardware,” the company warns developers. “Nanite will continue to expand its capabilities and improve performance in future releases of Unreal Engine.”

Lumen, meanwhile, is only designed to target 60 FPS for large outdoor scenes and 30 FPS for indoor scenes on the very latest console hardware. That’s a far cry from the 90 FPS minimum for most PC VR headsets. And with Quest 2 being significantly less powerful than the latest consoles, there’s just no way it’ll be able to handle those kinds of demands. Which may mean that the ultimate limitation in bringing these features to VR is simply performance (or lack thereof).

SEE ALSO
Vision Pro Preview: Early Thoughts on My Time Inside Apple's First Headset

The same scene rendered for a flat screen compared to being rendered for VR is often less performant in VR due to the need for stereoscopic rendering (and usually higher resolutions). Tricks like single-pass stereo and foveated rendering help reduce this overhead, but may not yet work in conjunction with the likes of Nanite and Lumen.

So it may be matter of optimization and more powerful hardware before it’s practical to bring these features to VR experiences. From Epic’s perspective, Unreal Engine has just a small fraction of VR developers compared to the likes of the Unity engine, on which the vast majority of VR games today are built. Especially with the trajectory of Meta’s Quest 2 as the most popular target platform for developers (and its lack of power compared to consoles and PCs), it seems likely that optimizing Nanite and Lumen for VR is very low on Epic’s priority list.

Hopefully we’ll see these next-gen engine features in VR eventually, but it might not happen for some time yet.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • kontis

    Last year they had a Lumen-dedicated stream and they said it wasn’t planned, at all, for VR.
    Not surprising considering it’s the main reason Matrix Awakens runs below 30 FPS at 1080p on PS5.

    Lumen also requires deferred renderer, so the VR optimal forward renderer (which Oculus and Valve pushed for) that supports MSAA will never have. This means even if Lumen was available for VR using it would result in blurry vision.

    Nanite should work, also on forward renderer, and the only reason it’s not yet available is because no one at Epic was delegated to plug it all together (that’s what they said).

    One of the rendering leads at Epic also said about VR:

    There are no signs that there even is a solution to locomotion. The metaverse means different things to people but it unquestionably means virtual worlds and without locomotion, VR can’t take part.

    • Yea, the same thought process is seen in their audio development team. More concerned in turning Unreal into mini synthesizer for generating dynamic tunes instead of advance spatial audio, or even updating Unreal (5) to Microsoft Window’s Spatial Audio API, which has been out for two years now. In fact the response I got from project leader is you should build a speaker system around you for spatial audio, you know like have a multiple curved screens. Sad, because I really like working in the Unreal Editor.

      • Hailey Ward

        I’ve received $18,590 only within a month and a half straightforwardly working small tasks from a computer.~sd405~Immediately when I lost my office career, I was exhausted but ultimately I have got this on-line project so I can manage to make thousand of dollars from the comfort of my home.~sd405~Each person can join this best career & obtain extra greenbacks on-line by viewing this web-link

        >>>>> http://­s­r­t­.­l­t­/t0k9s

    • alxslr

      There are no signs that there even is a solution to locomotion. The
      metaverse means different things to people but it unquestionably means
      virtual worlds and without locomotion, VR can’t take part.

      It’s true that locomotion is still the VR’s elefant in the room but, man! That’s such a closed mind statement!

      • Amelia Davis

        I have even produced $19,570 only in thirty days just working a few easy tasks from a laptop.~dq468~Immediately once I’ve lost my previous position, I was very distressed and thankfully I have discovered this best on-line opportunity that’s why I could make thousands staying at home.~dq468~Everyone can certainly join this best work & collect extra dollars online by checking this page

        >>>> https://g­u­r­l­.­p­r­o­/infolifestyles

      • Harper Rodriguez

        I have even collected $18,310 in no more than 21 days simply working parttime on my computer.~fw339~Just when I’d lost my office post, I was worried & thank God I’ve came across great online opportunity so in this way I’m capable to obtain thousand of dollars staying in home.~fw339~Any person can certainly join this golden opportunity & could get extra greenbacks on-line checking this web-site

        >>> http://­s­u­r­l­.­l­i­/bstze

      • Aubrey Scott

        I’m able to collect 85 dollars/h for doing job with a laptop.~A128~I have never realized that it could even possible but one of my good mate obtaining $27k just in a month by working this offer as well as she had influenced me to try.~A128~View further guidance on visiting this article
        >>> https://­f­.­l­s­/jobsavailable

    • Spectre

      To be honest, I think forward rendering + MSAA for VR is over-hyped. I’ve done some tests in Unity with a deferred pipeline + SMAA, and it was just as sharp and much cheaper to render than MSAA. I haven’t tested the new upscaling techniques like DLSS or FSR yet, but they may be even better in terms of quality for cost. Basically, as long as you’re not using TAA you shouldn’t need to worry about AA fuzziness.

      • Jesska

        This! I’ve put together a scene using both best practices and forward rendering + MSAA and Deferred rendering + DLSS.

        DLSS doesn’t work yet with forward rendering and so I basically had both better performance and better visuals with deferred+DLSS

        And performance was fine, it’s not like forward+MSAA was saving a lot to begin with even without DLSS.

  • Great article Ben and spot on.

    From a practical standpoint as a developer who has been using UE5 for awhile, there are other things that make UE5 VR unfriendly. Notably the removal of displacement textures in material in favor of virtual height maps. Since I use the landscape tool in “Apollo 11: ‘One Small Step For…’ VR Experiences” and the upcoming “Ingenuity in VR” (both available for free), displacement textures with distance control, provide that extra realism in small object depth that can’t duplicated by normal mapping in VR. Since it has been deprecated, not only does it cause all of my landscapes to go flat, virtual height maps are harder to set up and leave a lot be desired in parallax mesh anomalies, height control and glitches if you move too quickly. This is why 4.27.2 isn’t going to go away anytime soon.

    However that being said, UE5 nanite and lumen are fantastic for really nice real-time (30-60 fps) lighting, shadows and reflections without the pain of lightmap baking. I have used it to good effect with the cube renderer camera to capture stereo 360 images at a rate faster than other alternatives with good results. But yes, it seems visuals like audio, have not seen the updates to VR/XR in awhile like those seen in standard 2D gaming and the TV/Movie/Advertising industry.

    • Archpriest of Syrinx

      I have used it to good effect with the cube renderer camera to capture stereo 360 images at a rate faster than other alternatives with good results.

      You have used Nanite and Lumen together and produced a cubemap image in Unreal? How did you do this? I thought this was unsupported for Cubemaps

      • Sorry for the delay, but with UE 5.1 prerelease now out, anyone should be able to do it. Most of my cutting edge visualizations are done in Github ‘main’ builds. So features not yet available are being used. I also create my own material instances so I have more control over image projection. It also helps to have lots of VRAM too. I now use my slower RTX3060 more than the earlier RTX3070 due to it having 12 versus 9 gigabytes respectively. This is true for using GPU based lightmass baking, which I am starting to use in a hybrid lighting scheme with Lumen realtime VR in ‘Main’ build 5.2.

  • Cless

    People called me crazy when I said none of that would be out in quite a while, specially nanite. Got downvoted to hell anyways.

    • Ragbone

      That’s crazy.

      • Ragbone

        I’m crazierer

    • Jerald Doerr

      I don’t understand what you mean.. It came out in a beta form a while ago, and it’s out now with five?? So??

      • Ragbone

        That’s crazier.

      • Cless

        We are talking for VR here. As far as I’m aware, they don’t work for VR.

  • tomchall

    when are VR people going to understand that epic just simply isn’t interested in XR anymore as a company?

    • Andrew Jakobs

      When are people like you gonna understand that VR requires WAY more power and even for 2D flatscreen Nanite and Lumen is already running at 30fps or less on current midrange to highend hardware.. So you’re really unrealistic to expect them to have it running in 90fps on current VR hardware, Nanite and Lumen will work in VR if you really want it, but then expect it to run at 10fps or even 2 fps..

      • Spectre

        It’s not just that Nanite/Lumen aren’t supported in VR though, it’s that Epic hasn’t really done much of anything to move the needle on their VR tech since the forward renderer was first introduced (which was developed for Robo Recall — a launch title for the first consumer HMD ever!) They did release the VR template not long ago, so kudos for that, but… a little underwhelming considering all the other fancy new features that have come to the engine.

        Hell, I’ve been waiting for DFAO to come to VR for years — it’s literally a TODO comment in the rendering code — but they just never bothered. I tried hacking it in myself and got far enough to prove that it can almost certainly hit the performance targets on a mid-level machine, but I just wasn’t familiar enough with their rendering infrastructure (which is pretty arcane to me, but then again I’m not a rendering engineer) to get it working correctly.

        Another huge and relatively trivial improvement would have been a SMAA option for the deferred pipeline. The Astronauts patched it in for the VR version of The Vanishing of Ethan Carter, but it never made its way into the mainline branch, so forward rendering is basically mandatory, which frankly strips the soul out of Unreal Engine.

      • polysix

        Not only (soon) will VR be do-able with features like this due to better GPUs, but the HMDs themselves with foveated rendering should, in theory, mke them EASIER not harder to run on… so instead of 1 x full 4 K frame on a normal game, you get 2 X ‘small section where eye is looking at 4k equiv and the rest at low res’ … VR is actually going to be the answer to performnce, along with DLSS etc.. but if EPIC don’t get it working NOW then VR devs such as me can’t even plan and prototype for the future.

        EPIC should be very clear on VR and their intentions, more and more devs each day are turning their passion to VR after decades of same old 2D gaming… we are just waiting for better HMDs and PROPER VR ENGINES… UE should be that engine as it’s stunning in every other respect!

        (I’m a UE user since the first one in 98… 22 years ffs and I’m calling it… I love this engine but they MUST address VR ASAP!)

  • blue5peed

    Nanite in VR would be mind blowing. I hope other studios find a way to integrate a solution like this in their engines because Epic apparently is not so keen on VR.

    Although I do feel like geometry is in a good place on PCVR can you imagine Nanite on mobile? That could bring mobile games up to par with pc as far a geometry is concerned anyway.

    • Jerald Doerr

      Problem is high polycount for models takes up a lot of space.. Cell phone game like to be really small as a lot of people cells don’t have space.

      • Jesska

        Ease of development for nanite where you no longer need to create or tinker with LODs, or LOD distances, etc – essentially turning every mesh into nanite is a no brainer. Even if you build the world with simple mobile friendly few hundred triangles per mesh, nanite still provides huge benefits both to development and performance.

    • Andrew Jakobs

      It’s not that Epic isn’t keen on VR, it’s they are very realistic about the power needed. Even Nanite and Lumen can bring the highend GPU’s to its knee’s in 4k @ 60fps. Where is your sense of reality? Even current games already have problems running VR properly on highend systems without having to resort to things like timewarp or foveated rendering.
      Technically there is no problem using Nanite and Lumen in VR, but performance wise is where the problem lies, with current hardware it’ll run at 15fps or less on current headsets, if even that.

      • blue5peed

        Lumen is too intensive yes I agree with that premise but Nanite (virtualised geometry) actually gives back performance and dramatically so. It is less intensive than traditional methods not more. There are many benchmarks you can find on YouTube demonstrating this on UE5. At least that is my understanding from everything that I have seen and my own experimentation with UE5.

        It also uses less space or not as much as you with think due to the way nanite compiles data. so no you won’t be downloading 200gigs worth of assets onto your quest the Matrix city demo for example is around 17 gigs for a high-fidelity (never before seen fidelity really) open world city with NPC’s and cars included. The main downside is that so far it does not work with skinned meshes, so only hard surfaces.

        These facts are why I am bumbed that Epic hasn’t taken the time to port it to VR. It’s a priority thing not a technical limitation. It really makes me sad. I hope the rest of the industry catches up with their own virtualized geometry solutions soon.

        Lumen and Raytracing in general still have a long way to go so I’m not holding my breath for them, not this generation anyway. We may yet need another breakthrough before we can even consider it for VR. But Realtime PBR techniques should tide us over well enough.

        • kool

          Hopefully with psvr2 using ue5, the Sony optimization wizards will help get nanite working well in VR. Mark Cernys team should be involved in what ever their doing at Sony with vr since he designed the PS5 with vr in mind.

      • David Essentially

        They’ll need to figure it out at some point otherwise a massive chunk of market share will be out of reach and someone like Unity will come along and take a slice of their market share. I get the technical challenges but that’s not the developers or consumers problem, that’s an Epic problem they need to figure out. Lumen and Nanite are great and Unreal is def the leader in game engine design, but with the key selling points being Nanite and Lumen, having a gaping capability hole like VR will hold them back. Especially with Mark Zuckerbutt pushing Metaverse.

        • Andrew Jakobs

          But even without Nanite and Lumen UE5 has many advantages.

          • like loosing displacement mapping?

          • polysix

            I’ve used unreal engine since 1998! when it shipped with Unreal… I’ve mapped for years, then programmed, the moved to VR and was going great with UE4.. Unreal engine is nice to use and awesome for many things, UE5 is amazing for virtual production (Videos that look photorealistic instead of shooting in real locations).. however, making VR take a back seat is disapointing from EPIC who were always at the cutting edge (other than when Crytek leap frogged them for a while back around 2010).

            As a VR DEV.. if UE5 can’t bring these features to VR and seem to be ignoring it while VR is only ignored due to lack of engines working great for it… I’d have to move elsewhere.

            VR is the future… EPIC are fools to ignore that through, apparent, laziness or narrow minded ‘real thinking’ that doesn’t look ahead 5 years when most of our nanite/lumen VR projects would be on RTX8000 + foveated rendering and more than handle it!

          • Andrew Jakobs

            But another problem could just be resources within EPIC. I suspect that other parts of the engine used for things like flatscreen games/virtual productiion have precedence over VR which is still just a small part of usages. Even though I’m a VR fan and these days mainly only play VR games, I’m realistic enough to think that other parts have precedence over VR.
            But then again, some devs can take the code and try to see if they can fix the problems why nanite and Lumen might not work on VR, and if they fix it they can ask EPIC to integrate the fixes into the main branche.
            And Crytek also seems to have abandoned VR, I haven’t seen any games using the CryEngine outside their The Climb 2. It’s a shame as I hoped they just upgraded the engine and would ‘remaster’ the IMHO excellent VR game Robinson the journey, as I think that game would benefit from an update to natively support headsets beyond the original Vive/Rift (Yes it works on newer headsets, but you see it’s not rendering at native resolution).

    • Ragbone

      Doesn’t real life use Nanite?

  • AschVR

    Let’s say it explicitly: VR is for Unity.

    • Yen

      Hope don’t. I don’t want PS2 graphics. UE4 gave us a lot of incredible videogames and hope it will bring us a lot more.

      • Spectre

        Unity actually looks pretty damn nice nowadays with HDRP. The problem is that the infrastructure that enables all of Unity’s render pipeline flexibility also makes those pipelines relatively slow on the CPU-side, so you’re kind of forced to turn down the graphics options to hit performance targets. If they could shave just a few milliseconds off of HDRP’s CPU cost, Unity could be frankly stunning in VR.

    • alxslr

      6 years learning Unreal and I’m considering now making the jump. I never thought I would :-((
      I’ll give them another year, though..

      • asdf

        learn both… nothings stopping you

      • polysix

        24 years with unreal engine here! since “Unreal” (1998)… I’m massively into VR and this is annoying AF to see EPIC force VR into ‘also ran’ territory… I expected better from them, but since fortnite I guess they don’t care so much :/

    • Andrew Jakobs

      uhh, because UE5 Nanite doesn’t work for now you think VR is for Unity? UE5 still has many improvements over UE4 in regard to content handling even without Nanite. Unity will have exactly the same problems if they ever can implement something like Nanite or Lumen..

    • polysix

      Unity is shit… everything from it looks like a cartoon.

      UE has to get better for VR. It’s THAT simple! No excuses. VR is the future.

  • alxslr

    Especially with the trajectory of Meta’s Quest 2 as the most popular
    target platform for developers (and its lack of power compared to
    consoles and PCs), it seems likely that optimizing Nanite and Lumen for
    VR is very low on Epic’s priority list.

    Lumen is clearly far away, but for Nanite, that priority list could easily change if PSVR2 becomes a great success (which I think it will be).

    • Okazuma

      with the lack of Ps5, I doubt it will sell more than Quest 2

      • Corellianrogue

        It’s already sold almost 20 million in around 18 months, despite shortages. So there are plenty of PS5 owners to sell PSVR2 to plus maybe there won’t be a shortage, or as much of a shortage, when the PSVR2 comes out.

        • Okazuma

          Sony has sold 120 million PS4 and 7 million PSVR.
          How many PsVR2 can Sony sell on 17million Ps5?

          • Corellianrogue

            The original PSVR was a Frankenstein VR system (it required the PS4 Eye to use it at all and the Move wands to use it fully) with worse tracking than the PC (and eventually Oculus Quest) VR competition on a console that could only barely run it. And there weren’t 120 million PS4s when it was released, there were 45 million.

            The PSVR2 however is not only an all-in-one package but it’s relatively state of the art. Certainly better than most VR systems currently available. Although we haven’t seen the tracking yet I’m fairly confident that it will be at least as good as the Oculus Quest’s tracking. Plus the PS5 is far more powerful than the PS4. Definitely on par with a decent VR-capable PC and has the benefit of dynamic foveated rendering which will make that power go even further as its VR games will require less graphics rendered at full quality. It should have many more VR games from launch too (especially if it turns out to be backwards compatible) and Sony have asked developers to make all their PS5 games VR compatible if possible.

            Also, VR has gone pretty mainstream now with the Oculus Quest 1 & 2 so more people are aware of it and more likely to have tried it before, compared to when PSVR was released in 2016.

          • kool

            Psvr2 will sell a whole lot better because of the quest 2.

  • Ragbone

    Hopefully there wll be some fan made VR mods of upcoming games using this engine so us with GeForce RTX30 Series GPUs can enjoy these nice graphics.

    • Ookami

      I don’t think even RTX30 series GPUs will be able to handle either nanite or lumen. I don’t think any current hardware can run it. Might have to wait for something like RTX50/60 series–if we’re lucky RTX40, but I doubt it.

      • Ragbone

        I have modified my card, i cellotaped a voodoo 2 3dfx to it and put on a sticker of chuck norris so it will run fine.

        • Ookami

          Should work just fine in that case. I think ole Chuck could handle it on any GPU!
          More seriously though, I could be wrong about the requirements, I’m just giving my best guess *shrug*. Hopefully they can optimize it better.

      • Jesska

        Nanite doesn’t lose performance – you gain performance by using nanite in pretty much any scenario even with simple meshes. Especially for VR where number of drawcalls is a major limiting factor. With nanite you don’t have that problem at all.

        Lumen – yes. Lumen is extremely heavy. That won’t work for VR for a few years at least.

        But byproduct of lumen development is that they created much better mesh distant fields for ue5. So existing dynamic shadow maps will look significantly better with it and can be great for some VR games with dynamic lighning.

        • Ookami

          You might be right, but for whatever reason it doesn’t work in VR. Maybe it’s just a software issue.

  • Jonathan Winters III

    This article’s headline is inaccurate; the features aren’t “not ready”, they are not in progress nor planned.

  • polysix

    It needs to come to VR or it’s basically pointless… for a ‘future looking’ engine that we’re supposed to turn to for everything from virtual production to VR.
    I get that right now we don’t have the power but Epic should get it running even if it’s SLOOOW and soon.. an RTX4090 should be able to cope… if not then a 5090… and eventually mainstream cards, but some of us want to work on MULTI-YEAR-LONG projects that need the beauty and ease of LUMEN in VR to make it viable, even if we can’t release it for the public until 5 years from now when we have much faster PCs and even standalone VR that has perf improving foveated rendering and / or wireless PC streaming even better than today.
    I think Epic got lazy personally and just can’t be bothered to put the effort into lumen for VR right now as VR isn’t YET massive.
    Problem is, it won’t ever be massive if the cutting edge engines on the fastest systems (top end PCs) won’t even activate the features needed to make VR look as good or better than the best flat AAA games!