HoloLens Autodesk MotionBuilder App Lets SFX Artists Build Fantasy in Reality

2

As HoloLens filters its way into more developers’ hands, we’re beginning to see some of the true potential in Microsoft’s ‘Holographic’ augmented reality headset. This early new project from Jasper Brekelmans shows the potential, practical applications for SFX artists to build digital visual effects against reality in real time.

The separation of the digital and physical world is one digital artists in the entertainment industry have to tackle regularly. So what if there was a tool which allowed an SFX artists to build his effects and try them out, in real time, overlaid into reality?

Jasper Brekelmans thinks he has a solution, or at least the beginnings of one. He’s using Microsoft’s augmented reality visor HoloLens to build an AR integration with Autodesk’s popular MotionBuilder character animation studio.

hololens-autodesk-motionbuilder

“Virtual Production (the ability to see virtual 3D content through the camera lens while shooting) has become a standard tool in many big movie productions these days,” says Brekelmans, “This early prototype shows how to live stream a 3D character with motion capture data into the Microsoft HoloLens from Autodesk Motionbuilder [a popular professional 3D character animation tool], using a simple WiFi connection.”

See Also: HoloLens Enterprise Mixed Reality Development with Object Theory
See Also: HoloLens Enterprise Mixed Reality Development with Object Theory

The main thrust of Brekelmans’ project is to take SFX pre-vis to new levels. “Integrating a HoloLens in this process would allow a visual effects supervisor or director to see these 3D assets while walking around a live-action movie set, without the need for looking through a camera/tablet screen. Giving them a more natural way to get a feel for scene blocking and finding the best shot.”

Brekelmans has released a video (embedded at the top of this page), demonstrating the new project in action, filmed through the HoloLens visor, aptly demonstrating the HoloLens’ tracking abilities. “The HoloLens is a revolutionary device, it’s rotational & positional tracking are amazingly robust. It’s environmental understanding (it can detect where the floor, walls, ceiling and tables are for example) combined with being a fully standalone wearable Windows 10 computer open up possibilities that weren’t possible before,” states Brekelmans.

SEE ALSO
'Starship Home' Turns Your Room into a Flying Alien Plant Sanctuary, Coming to Quest 3 This Year
See Also: One Explosion and Three Launch Aborts Later, HoloLens Finally in Use Aboard ISS
See Also: One Explosion and Three Launch Aborts Later, HoloLens Finally in Use Aboard ISS

Of course, working with what is essentially just one step beyond a prototype device has its challenges. “Being a 1st generation developer kit has it’s limitations, interaction (gaze, speech & gesture) can be somewhat limiting, field of view is not extremely wide yet and being a mobile device there are CPU/GPU constraints,” he says. “Still it allows for experimenting with the future in the present day and just like we’ve seen after the first Oculus Rift release things will only improve in the next few years and iterations.”

It’s an impressive achievement, and seeing it in action really does show HoloLens playing to its strengths. Crafting fantasy characters and objects and at the same time understanding how they look once placed into the real world is a potentially extremely powerful tool, which is heartening to see given the scattershot direction MS appear to be taking with pitching the HoloLens up to now.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


Based in the UK, Paul has been immersed in interactive entertainment for the best part of 27 years and has followed advances in gaming with a passionate fervour. His obsession with graphical fidelity over the years has had him branded a ‘graphics whore’ (which he views as the highest compliment) more than once and he holds a particular candle for the dream of the ultimate immersive gaming experience. Having followed and been disappointed by the original VR explosion of the 90s, he then founded RiftVR.com to follow the new and exciting prospect of the rebirth of VR in products like the Oculus Rift. Paul joined forces with Ben to help build the new Road to VR in preparation for what he sees as VR’s coming of age over the next few years.
  • Oshriez

    Worth mentioning that the work is part of a European commission funded research project called WholoDance. (www.wholodance.eu)

    • Brekel

      Yes, the robot model and mocap sequence used in the clip are part of the WholoDance project indeed.