A new performance opened at The Shed’s Griffin Theater in New York this week, KAGAMI, which blends theater, music, film and Magic Leap 2 mixed reality headsets. The fifty minute experience is built around a volumetric capture of late composer and performer Ryuichi Sakamoto. This is a once-in-a-lifetime opportunity for New York audiences to see the mixed reality future of live performance. But you’d better hurry. The show closes July 2nd. New York will not see its like again anytime soon. Theater is hard enough without putting a $3,299 Magic Leap 2 XR headset on each of the 80 people in the audience. For this reason, the only similar performance we know of was a 2017 augmented reality dance performance at Case Western University, which was a HoloLens launch partner. It has never been done in New York.
Audiece members wearing the Magic Leap 2 mixed reality headset.
KAGAMI, which translates to “mirror” in Japanese, began production in 2020, when director Todd Eckert and Sakamoto traveled from New York to Tokyo to use a unique four-camera volumetric capture system to record him performing a select set of ten solo piano pieces from his catalog. Sakamoto’s work is notably genre defying. At some moments it sounds like classical music, at others, jazz and dance, then all-at-once larger than life movie music. He won the Academy Award for his work on “The Last Emperor.” One piece in performance is dedicated to the memory of that film’s director, Bernardo Bertolucci.
Ryuichi Sakamoto and director Todd Eckert on the capture stage.
Eckert was Magic Leap’s first content executive, joining the iconic AR pioneer in 2012, making him one of the new medium’s first practitioners. To create KAGAMI, Eckert took the 3D recording, or volumetric capture, of Sakamoto and put it inside an infinitely fungible 3D virtual world. Wearing Magic Leap headsets we’re invited into this world where we can walk around freely. Around Sakamoto’s performing hologram, which we can see from any angle, are special effects and images the music inspired Eckert to make. It envelops Sakamoto, and the audience, as he plays.
The late composer Ryuichi Sakamoto performs as a hologram.
As the show begins, we’re advised we can move around if we choose. No one does, at first. Sakamoto’s hologram, seated at the piano, materializes in the middle of the room and begins to play the first piece. A few of us rise and move toward the performer. Soon, others follow. Slowly, effects start to flow around us. Augmented by the theatrical lighting in the room, the music, effects, and images create a world moving and morphing around us. When Sakamoto, well, his hologram, ends a piece, there is silence. One person claps before they realize this is not a live performance.
In one piece, a tree grows out of the piano and its roots go far beneath it below the floor. It dissolves into stars, the milky way, and soon we find ourselves standing above the earth as seen from space. In another composition we’re surrounded by iconic New York images: skylines, bridges, unexpected terracotta lions. Other times, Eckert shows us black and white warplanes flying in formation and other iconic World War images.
Late composer Ryuichi Sakamoto.
Sakamoto died during the production, which was interrupted by the pandemic. The ghost-like appearance of the other audience members in mixed reality, the volumetric capture of the deceased Sakamoto, and the elegiac images chosen by Eckert imbued KAGAMI with the air of nostalgia. "Mixed Reality is still in a pretty nascent place, so the art direction was tricky.” Eckert said. “Everything began with the songs, though they’re all so beautiful it was important to determine what was specific to each composition and not just focus on the obvious. There’s an economical elegance that always defined his playing, and though I didn’t have a theme to the show I think that was present in every approach. I honestly expected him to be with us here for the opening - and the production took around five years so his health was not always an issue. But if there’s a sense of remembrance - and that makes me happy, honestly - it can most reasonably be attributed to him."
KAGAMI runs Tuesdays through Saturdays through July 2nd at the Shed Theater, 545 W 30th St. Tickets and showtimes. It will also be presented at the 2023 Manchester International Festival in the UK, continuing in 2024 to the Sydney Opera House and Big Ears Festival.
