The GRAMMY award-winning artist just set a new standard for livestream performances.
Edit (6/6/2020): We’ve updated the piece to better clarify how the live performance was conducted. Legend himself did not perform in VR. His movements were captured and transmitted to a digital avatar in real-time before being broadcasted to a remote audience tuning in via multiple livestreams.
Fresh off their huge $30 Million funding announcement, virtual music platform Wave hosted an amazing live concert last week featuring GRAMMY award-winning singer-songwriter John Legend in an effort to help awareness and support for Legend’s FREE AMERICA campaign, with the goal of transforming the U.S. criminal justice system and ending mass incarceration for African Americans.
During the 17 minute live virtual musical extravaganza, Legend performed a number of his biggest hits such as “Ooh Laa” and “Love Me Now.” He also performed songs from his latest album, Bigger Love.
Between songs Legend would chat with an audience made up of users tuning in via Wave as well as people watching from a livestream on Twitter and YouTube. Those in the audience could donate to the cause from their respective platforms, at which point their names would appear around Legend in the form of flowers, comets, and various other environmental elements.
Legend’s avatar moved gracefully has he moved about the virtual stage, his digital movements matching that of his real body with incredible detail. Even his passionate facial expressions were matched in real-time, adding an additional layer of intimacy and emotion to the performance.
Legend conducted his performance live in-studio wearing an Xsens MVN Animate Motion Capture System. This is the same motion capture solution used on blockbuster films like Star Wars: Rise of Skywalker, Avengers: Infinity War, and The Wolverine, as well as hit games such as Creed: Rise to Glory and Counter Strike.
This same technology is also being used in industries such as health to measure human machine interactions and the kinematic and kinetic analysis of locomotion (better known as gait analysis), as well as sports science to explore better and safer workout regiments for athletes, trainers, or those looking to hit basic health goals.
Looking at the video posted on John Legend’s YouTube page that shows both real-world and VR footage, it looks like he’s wearing Xsens’ MVN Link suit.
The MVN Link suit features embedded sensors and is used primarily for high dynamic movement situations, such as fast running, live on-stage entertainment, and live streaming. Data from the suit is then transferred to a computer through a WiFi connection at a high frame rate. The most impressive part of the suit is its on-body recording capabilities. This means you can capture data without a computer and then upload the data later; and because it’s a wireless system, the user has total freedom to move how they want.
Xsens also offers an additional mocap solution referred to as the MVN Awinda. Unlike the MVN Link, this wireless system is composed of a top shirt made of lycra along with straps that users wear on their arms and legs to capture data, allowing them to wear the suit above their normal clothes. This makes the suit easier to take on and off in case you want to capture data from different people. The information is captured at a lower frame rate, which makes the suit perfect for jobs that aren’t as high dynamic or complex.
To transfer the data from the suit to a computer, Xsens uses a software called MVN Animate, which allows you to record, view and analyse the data in real-time. That data is then instantly turned into a robotic avatar (lovingly called Mo by the Xsens team) that can be viewed on a computer or through a mobile device using the Xsens MVN Remote app.
The data can then be exported into all typical formats used to create visual effects such as FBX and BVH, and can be streamed directly into Unity, Unreal, and Autodesk, where you can then add custom visual details.
You can see a much more detailed breakdown of how the suit works, here.
During Legend’s live virtual concert experience, he steps away from the mic and sits down at a piano to play his hit, “All of Me.” The Xsens suit lets you bring in up to 4 real-world props into the virtual scene, including a piano!
In the end John Legend’s A Night for Bigger Love Wave concert was an incredible moment for VR and live music in general.
Wave has already produced numerous Waves (what Wave calls live shows) with a number of artists around the globe, and have more Waves lined up on their calendar.
If you want more information or if you would like to donate to Legend’s FREE AMERICA campaign, click here.
Image Credit: Wave
