Fully immersive experiences without the need of a VR headset. Are we there yet?
There are many tantalizing possibilities surrounding current VR and AR technology. Unfortunately, due to the expensive hardware and sometimes complicated setup required to engage in AAA experiences, these captivating digital adventures are often out of reach to the average consumer. This includes everything from pricey VR headsets, to high-end gaming PC’s required to run them.
In an ideal world, users wouldn’t have to rely on wearable tech or inconvenient hardware to immerse themselves in a digital world; all the virtual action would just simply happen all around you – just like real life. There are many companies working on various visions of a Holodeck-like experience, including Light Field Lab whom we covered recently, but nearly all remain at an early conceptual stage.
Recent advances in existing projection technology have, however, made possible a new kind of immersive movie theater experience, such as Space 360, which opened last year in Guangju, South Korea.
The setup inside this sphere, which spans 12 meters in diameter, consists of a transparent glass observation bridge that stretches across the entirety of the interior space. High-resolution projections displayed on the dome envelope users in 360-degrees, providing a captivating immersive experienceminus the need of a VR headset.
The project was financed by Korea Hydro & Nuclear Power Co. (the country’s national operator of power plants) which then proceeded to donate the entire operation to the National Science Museum as part of its corporate social responsibility program.
The first show created exclusively for the sphere told the story of the Big Bang and the beginnings of human civilization. The experience is available as both a fulldome experience and in VR.
Ukrainian creative studio Front Pictures was responsible for engineering and installing the projection system and software. The team has extensive experience engineering fulldome theaters and planetariums, having so far delivered over 150 such installations all around the world. The concept of a fully immersive installation, however, was more challenging. For starters, placing the projectors in such a way that they would not shine directly in the viewer’s eyes was difficult.
“We wanted to make a true VR theater that would give visitors a breathtaking 360 immersive experience. But to meet our idea of the future, the system also needed to have the highest possible resolution and brightness of projection, high energy efficiency and cost effectiveness,” they recounted in a blog post on their website.
“Unlike in a conventional digital planetarium, where projectors are located along the dome’s perimeter, the options for placing them in a full sphere are substantially limited. One of the challenges was to devise a layout that would avoid shadows being cast by the observation bridge, minimize openings for the projectors, while, at the same time, maximizing the resolution and brightness of the projection.”
In the end, Front Pictures chose to use 12 projectors located around two entrance doors. Engineers conducted multiple tests to discover a layout that would allow for rotation at extreme angles, as well as different equipment configurations offered by various vendors (noise was also a big factor considering the number of projectors in an enclosed space.)
“The spherical screen creates significant geometrical distortions of the projected image. The distortions become even larger due to the fact that in a full projection sphere, it is impossible to place projectors along the perimeter, as a sphere has no edge. They need to be located around the entrance doors on the opposite sides of the sphere.”
A conventional fulldome experience consists of only half a sphere, so one can normally get away with deploying a single fisheye lens. This is not enough to completely cover a 360-degree spherical screen with a bridge in the center, however. In this situation, manual calibration – i.e. stitching the images together isn’t the best option, so the team instead adopted an automated calibration system based on powerful computer vision and image analysis algorithms combined with a 360 degree multiple camera rig. Front Pictures has been developing auto-calibration technology since 2012 which analyses patterns to seamlessly blend projectors together into a unified coordinated space.
Front Pictures used four cameras with fisheye lenses mounted around the glass bridge, customizing their software to enable the multi-camera calibration.
“The previous generation of multi-projector systems were built around the “one computer per one projector” paradigm,” they continue via their blog post. “But this approach has significant drawbacks such as a relatively big budget for computer hardware, more possible points of failure, difficulties with implementing proper failover solution, as well as higher electricity consumption. Another downside of the cluster systems is a discrepancy in video playback speed on different computers, which can result in rough, jerky video playback.”
To address this particular issue, the team developed a technology called Screenberry. The hybrid video processing engine utilizes both the GPU and CPU in a much more efficient manner, resulting in smooth 8x8K video playback on up to 72 output devices; all of which connected to a single computer.
The entire operation highlights just how incredibly difficult and time-consuming fulldome technology can be. However, continued market growth shows that there is in fact an appetite for these types of headset-free experiences, it’s just a matter of finding the best way to deliver them. Whether or not that will involve any sort of wearable device remains an open question.
Image Credit: Front Pictures
