Prince thought the idea “demonic” but holographic proxies for pop stars are closer to the bone than ever. In a claimed world first, a single by Brazilian chanteuse Laura Rizzotto has been released, accompanied by an AR ‘holographic’ app. Fans can see her performing the track in virtual three-dimensionality anywhere they point their phone.
An augmented reality app in and of itself is not groundbreaking, but “the holographic performance we are premiering with Laura Rizzotto absolutely is,” according to its makers, the LA-based volumetric capture studio Metastage, quoted in Forbes.
Rizzotto’s performance was filmed with 106 video cameras - 53 RGB cameras and 53 infrareds – shooting 30fps, although the rig has 60fps capability. The data was passed through the Microsoft Mixed Reality Capture software to output a 3D asset that Metastage claims maintains the integrity and fidelity of the original performance.
Anything ‘first’ is guaranteed publicity so this could be dismissed as a publicity gimmick for a singer-songwriter who is not exactly Beyonce-famous.
On the other hand, the dystopian scenario of Charlie Brooker’s recent Black Mirror episode, in which Miley Cyrus’ singer Ashley O was replaced with a hologram, is but a hop, skip and a step from coming to pass.
We’ve had holographic resuscitations from Tupac to Roy Orbison, completely (hollow) graphic pop-stars like Japanese pop idol Hatsune Miku and fictional social influencers like Lil Miquela (1.6 million Instagram followers and counting), but the concept of an actual artist being able to multiply their presence and therefore their revenue by materialising at any venue in the world (simultaneously) must be music to the ears of record company execs.
Cold Play might find it a useful way of cutting down their carbon footprint on tour. The Rolling Stones might like to consider it as a way of crystallising their own carbon imprint before extinction.
After all, one of the business goals of Metastage is to create a digital archive of holographic assets that can be preserved and enjoyed (played back) forever.
“For the first time ever, users can tailor the artist’s performance to their own taste and comfort,” says Carolina Rizzotto, co-producer of her sister’s Metastage Experience. “You can take Laura and choose where she’ll perform and even change her capture size. You can make her twice her normal size...or you can even make her the size of an ant and have her perform in your own hand.”
Twice her normal size – that’s eerily reminiscent of the Black Mirror episode too.
For this to be streamed on a mobile device, with or without 5G, the file size needs to be manageably small and so it is with Metastage claiming the Microsoft process can compress the 10 gigabytes of data it captures per second down to 10 megabits a minute of volumetric capture. Arguably that’s ok on a phone where HD rather than 4K video, provides a good enough experience.
Metastage’s app is developed in partnership with Magnopus (the company behind immersive experiences for Pixar’s Coco and Blade Runner: 2049). It suggests that end-to-end production costs around $10k per second – not excessive compared to high-end commercials - but will rise depending on the photorealism required.
Unlike traditional pop promos, performers can’t yet rely on an edit of the volumetric capture to mask any failures in performance or dancing skill. The capture zone itself is confined to an 8-foot-diameter circular area. Rizzotto had to get the entire choreography right in full, uninterrupted takes.
“There is no editing to the (volumetric) performance, as it has to be in one take only, which makes it even rawer,” according to Carolina Rizzoto.
Carolina, incidentally, previously produced a short educational film called ‘The Future is Here - How Sci-fi Becomes Reality’, starring George Takei (Sulu).
The most important take away from this fun development of holography is the idea that we might all be able to own 3D avatars of ourselves in the emerging three-dimensional internet. Operating a hologram is only a step removed from the 2D online presence we now portray of ourselves on social media.
The chief executive of Metastage, Christina Heller, talks of the ability for volumetric capture to overcome the ‘uncanny valley’ of the inauthenticity of existing animated human techniques.
“When you watch something that was captured volumetrically, you know that it was captured live. It wasn’t reanimated,” she said in an interview with XR for Business. “It wasn’t puppeteered. This is something that you can watch with the same awe that you would a live performance happening right in front of you.”
The issue – whether you are a pop star, an actor (alive or dead), or plain Joe Public - is who owns our digital selves? In a world in which deep fakes are in many cases indistinguishable from the real thing, who protects our avatars on the 3D internet?