Imagine standing in Peru on the edge of the highest peak on Machu Picchu, sitting in the front row of a Chanel fashion show at the Palais Royal, or witnessing the adrenaline rush of a gold medal sprinter at the Olympics in Rio. Virtual reality (VR) makes these types of experiences accessible from your couch. It’s an incredibly exciting tool that goes beyond mere entertainment or gaming, and it’s even more exciting to be involved at these early stages – freely experimenting and collectively building best practices for this emerging medium.
In order to realize VR video’s true potential, content needs to be captured at ultra high resolution so that image fidelity in the headset feels fully immersive and photoreal. As we know, most VR video content today looks surprisingly soft and blurry. Even when the full 360-degree image is captured in 4K, your field of view only covers one small section of that. These pixels then are stretched back out to match the headset’s display resolution. This means we need to start out with content captured at a much higher resolution, so that the final field of view can align with the display resolution without losing quality.
Working with the premiere camera manufacturer RED and production company SuperSphere, the Pixvana team embarked on a live action shoot in Los Angeles. We captured content at 10K resolution and 60 frames per second (FPS) with the end goal of preserving as much image quality as possible throughout the post-production pipeline for a truly high-res, lifelike VR experience. Along the way, we learned some critical lessons about the challenges of facilitating production of such high-res content in VR.
Our shoot was directed by VFX veteran and Pixvana co-founder Scott Squires, and covered four iconic LA locations over two days including the theater at the Ace Hotel and the Los Angeles Arboretum. The locations were chosen based on environment and architecture, leveraging the high resolution to showcase as many interesting details as possible. We teamed up with strategic creative agency Bread n Butter to produce this live shoot, incorporating couture dresses from fashion designer Ralph & Russo and high-end fashion models to add character and creativity to each location.
One complication from shooting at this high resolution is that we had to assemble a custom rig. Most rigs available today shoot in HD or 4K, but not beyond, and they’re automated so you can’t fully control the exposure or swap out the lenses. The Diamond Brothers (twin VR DP’s Josh and Jason from SuperSphere) built their custom Diamond Dragon rig out of five 6K RED Weapon cameras in a one-up and four-around configuration, each equipped with a Canon 8-15mm L fisheye lens to let us capture a wider view per camera. This rig gave us the resolution and customization we were looking for, including large dynamic range, tailored lenses, and ability to easily sync and genlock the cameras.
Due to the substantial size of the RED Weapons, it was necessary to design a layout that brought each camera as close together as possible and lessen parallax. The actors and other subject matter needed to be at specific distances from the rig for proper capture and stitch. This wasn’t a huge roadblock on our shoot, but is something to be aware of as a director plans their shots and shooting plan. Any multi-camera rig must always balance size and weight. The RED Weapon was definitely the best choice for the resolution we wanted, but five of them on one rig made for a larger system that took planning and effort to move. In order to capture ultra high resolution video, for now VR content creators will need to evaluate the unique needs of their shoots and plan accordingly.
We also experimented with how to improve stitching on-set in order to make things easier in post. The fisheye lenses gave us a generous amount of overlap between neighboring cameras, which allowed for flexibility on where to place the stitch seam. We also carefully framed the action within the main front-facing camera as much as possible. Scott and I quickly learned that a VR shot is just like a typical VFX shot – you won’t fully be able to tell what the shot looks like until you get into post, but some quick on-set stitching would let us know whether the footage was on the right track. Without a live-stitching tool readily available today, it’s imperative to have someone on-set who can do these quick realtime stitches as needed.
Over the two-day shoot, we captured a total of 70 minutes of footage, which at the high resolution amounted to over 7 terabytes of data. Our advice to those working in ultra hi-res VR video: it’s hard to overestimate the storage capacity that you’ll need, even for short form projects. Be prepared!
Ultimately, VR video production workflows are still in their infancy. Though we’re able to find solutions and improve efficiency here and there, the workflow truly needs to be rebuilt from the traditional cinematic perspective to facilitate this new and unique medium. At Pixvana, we’re working on ways to make that happen by developing new tools for VR video creation and delivery. We continue to research best practices for capturing ultra high-res VR video, and look forward to sharing more lessons learned from future shoots.
