Nickelodeon let you talk to Michelangelo and Donatello.
Select fans got a surprise they weren’t expecting when they put on a VR headset to check out the latest Teenage Mutant Ninja Turtle experience at Comic-Con. Instead of just dropping into a pre-recorded VR experience, users got to conduct a live interview with Mikey or Donnie while in virtual reality.
Dubbed the “Rise of the Teenage Mutant Ninja Turtles Live” experience, the one-of-a-kind opportunity let you chat with your favorite turtle heroes in real-time and take home a mixed reality video of yourself composited into the interview with a CG Nickelodeon avatar as your head.
Fans lucky enough to try the experience stood against a green screen and picked from one of six Nickelodeon characters – Lincoln Loud, Luna Loud, Sandy Cheeks, Invader ZIM, or Angelica Pickles. The character was the your head, while your actual body was in the virtual world, immersing you in Turtles’ iconic New York City world.
The conversation you had with Mikey or Donnie was voiced live on the scene by Brandon Mychal Smith and Josh Brener, respectively. A combination of software was used to execute the endeavor, including Adobe Character Animator, Unreal Engine, and NewTeK NDI.
Built entirely in-house at Nickelodeon’s Entertainment Lab, using a game engine allowed the ability to create real-time interactions and conversations. The experience streams in Mikey and Donnie puppets by using NewTek NDI Technology. The Mikey and Donnie puppets were created and are driven live using Adobe Character Animator. MIDI Keyboards, with mapped animation cycles, are used to trigger the various poses Mikey and Donnie can do. These MIDI Keyboards are puppeted live during the interviews. NDI allows the puppets to be streamed out of Adobe Character Animator, and picked up by various other machines which use these live animations for the experience and compositing.
The creation of this experience took six months. The Entertainment Lab worked with the Rise of the Teenage Mutant Ninja Turtles production to bring their uniquely stylized environment and characters into an immersive virtual reality set.
Having attended Comic-Con consistently over the years scouting the best in VR, one thing’s for sure, this was definitely the year VR took huge leaps forward. From the advanced wireless motion capture experience we saw for Jack Ryan and now this real-time TMNT VR experience, we can’t wait to see how crazy next year’s Con is going to be.
