A team of researchers bring 2D photos to life with automatic 3D animations.
As AR technology continues to develop from past its current state of infancy, one of its most popular use-cases so far has been the digital enhancement of real-world images. Through the use of various techniques, such as hidden AR markers – which, when scanned with a smart devices camera activates a supplementary augmented experience – everyone from artists to corporations have been utilizing the technology to add a whole new dimension to their projects.
With Photo Wake-Up, a new form of 3D character animation, a team of researchers from the University of Washington have expanded upon the idea of augmented reality-enhanced photos by introducing a technique that can convert a 2D image into a 3D animation which, with the help of AR technology, can literally walk into our reality.
Image Credit: Chung-Yi Weng, Brian Curless, Ira Kemelmacher-Shlizerman
Using a a program referred to as SMPL in tandem with deep learning, researchers devised a system capable of cutting out the subject from a 2D image, rigging it to a prebuilt skeleton, and then animating it with a series of predetermined moves, such as running, walking, and jumping. When used with a smart device, such as a smartphone or tablet, the 3D animation can then physically “wake-up” out of its 2D form and walk into reality. See what I did there? The team, composed of Chung-Yi Weng, Brian Curless, and Ira Kemelmacher-Shlizerman, used a collection of over 70 images they obtained online, ranging from Steph Curry, Beatles cover art, Ironman, even art from the legendary street artist Banksy.
“Our method works with large variety of whole-body, fairly frontal photos, ranging from sports photos, to art, and posters,” states the team during an interview with MIT Technology Review.
“We believe the method not only enables new ways for people to enjoy and interact with photos, but also suggests a pathway to reconstructing a virtual avatar from a single image while providing insight into the state of the art of human modelling from a single photo.”
The team has provided a paper detailing the specifics of their technique, as well as a variety of supplementary information.
“We present a method and application for animating a human subject from a single photo. E.g., the character can walk out, run, sit, or jump in 3D,” states the papers abstract.
