Replay: Here's one very good example by the RAF of how 360 filming can be used to make a truly immersive experience video using Insta360's Pro2 system. 360 video is full of logistical and technical hurdles, and this is the story of how they did it.
Early in 2020, the RAF Police approached RAF Media Reserves for specialist help creating a 360 degree immersive video explaining their role. They wanted to do something different that would wow and engage an audience, and it was decided that immersive VR video was the best way to achieve this.
The brief was deceptively simple: “We want a five minute immersive video that shows what we do. You’re the media specialists, so over to you to work out how to do it.”
On the face of it, it was no different to the many corporate video briefs that we’d worked on in our civilian or RAF lives, and the process of drilling down into the brief to discover what the client really wants or expects will be familiar to many reading this. Our mantra while putting together the final storyboards and then a working script was that there was little point in using immersive technology simply to do what a conventional video could achieve just as easily.
So our starting point was to find RAF assets and locations which could fill the frame of a static 360 camera and offer video and audio interest not just around the viewer but above and below them as well.
What became evident early on is that there are remarkably few examples of 360 video in which actors deliver scripted lines and work to choreographed actions and marks. Yet this is exactly what we needed to deliver if we were to use immersive video to tell the RAF Police story. We wanted the ‘realism’ of a presenter telling the story and guiding the viewer rather than a disembodied voiceover.
For the final production we settled on five key sequences, each with their own individual hurdles to overcome.
We were acutely aware of the need to capitalise on the initial ‘wow!’ factor, so we set our first scene in the cavernous empty hold of an RAF C-17 transport aircraft. As RAF reservists, we’d both experienced a sense of awe the first time we’d walked up the ramp into the belly of one of these beasts.
Our kit for most of this project comprised an Insta360 Pro 2 and One R, Zoom H3-VR ambisonic recorder plus radio mics and recorders. Our lessons in the practicalities of 360 filming began the moment we set up the camera in the C-17 hold.
There's nowhere for the crew to hide from a 360 camera, which means that monitoring of the shot must be done remotely via a tablet or smartphone which, by its very nature, can only ‘look’ in one direction at a time (albeit pretty wide angle). In turn this means that it is easy to overlook something on an apparently successful take that only becomes evident in the edit.
For the C-17 shot we were able to position ourselves next to the ramp just outside the aircraft fuselage, but the solid metal construction didn’t do much for the range either of our radio mic or the Pro 2’s built-in Wi-Fi monitoring (nominally 20m). We were glad that we had the Insta360’s Farsight wireless monitoring system that gave a strong and reliable signal throughout all of our scenarios.
The nature of 360 filming would mean that our presenter, RAF Police Flight Sergeant Neil ‘Josie’ Wales would have to memorise his lines, and we needed to get each shot complete in one take. There would be no room for cuts or any of the usual trickery reserved for covering glitches.
Josie not only had to memorise and correctly deliver lines, but also hit marks on the floor to give the viewer a full 360 experience as he walked around the hold, all while never getting closer than two metres to the camera, which would make stitching the scene together very difficult or impossible. We also had to ensure that Josie didn’t stand in the ‘stitch line’ of the lenses.
In another scene the RAF military working dog handler’s route was calculated to draw the viewer’s eye, and the discussion going on between the RAF police over a 4x4 vehicle’s bonnet played to the scene’s narrative. But the real purpose of the carefully placed vehicle was to provide a hiding place for the film crew. This was the big 360 filmmaking lesson for this scene.
Using a larger iPad rather than smartphone to view a 360 equirectangular image helped to spot problems, but the large amount of detail crammed into a relatively small screen still means that some things only become obvious when it’s too late to change them.
In this case, it was only in the edit that we discovered our sound recordist’s feet sticking out from under the police 4x4. They had to be removed with some deft rotoscoping in which our presenter was separated from the background, the background plate fixed by painting out the recordists' legs, and then the presenter placed back on top. Spotting this ‘elephant trap’ at time of shooting would have saved several hours of editing!
A scene that made maximum use of the technology involved a C-130 Hercules on final approach. Recording an aircraft coming towards the viewer makes them look up (or perhaps duck!) as it goes overhead, then turn to look behind them as it disappears down the runway.
One lesson from such an exposed airfield environment was the tension between trying to keep the tripod footprint small and the need to steady it against the wind. Tripod and camera shadow was also an issue, requiring a balancing act between trying to prevent it altogether and accepting the need to remove it in post.
In another sequence we really wanted to shoot at night for the atmospherics bestowed by floodlights and flashing red and blue emergency vehicle lights. However the small sensors on the Insta360 Pro 2 results in relatively poor low-light performance. We therefore decided to shoot the scene twice – once in daylight as a safety shot and then again that evening. Luckily the night scene turned out very well, helped in no small part by the use of the Neat Video plugin, which did a great job of removing the noise. Without this the images were on the verge of being unusable. Even so, the time in post to denoise the 8K ProRes 422 files, even on a 12-core PC with an nVidia RTX 2070 Super GPU was considerable.
This is the scene in which the quality and complexity of the sound edit is most evident. Ensuring that the Zoom H3-VR ambisonic mic was aligned on the same axis as the camera helped the correct spatial positioning of the edit. On top of that, several separately recorded tracks and some sound effects were also positioned in the 3D audio mix.
You can watch the finished VR experience using this link on YouTube with suitable headsets, or via Oculus TV. The YouTube app is required to view 360 on mobile devices.
In the next part Tony and Paul take us through the mammoth task of putting the sequences together in post production.