From cameras on: the tools you need to make VR

Facebook's open source VR pro-level camera, Surround 360

As VR tips over into the mainstream, the tools necessary for serious virtual world and experience building are finally emerging. From cameras to editing kit, Mark Ramshaw details what’s available.

As Academy Award-winning vfx artist and filmmaker Robert Stromberg notes, one of the key challenges facing storytellers adapting to the virtual reality medium is “learning to think in terms of spheres instead of rectangles”. Until recently this was further compounded by the lack of plug-and-play hardware and software designed to handle those additional dimensional considerations.

Standard and VR filmmaking may appear to be close cousins, but the simple act of dropping the audience directly into the action - with viewers able to look anywhere at any given moment - changes everything. Traditional filmmaking rules and grammar no longer apply: there’s no shot framing, no fourth wall, and no way to firmly focus on specific areas in a scene, and that’s just for starters. Even visual effects require a rethink, for instead of designing CG elements for compositing into the scene based on the camera view, artists instead need to ‘world build’ - creating elements to be viewed from any position and distance.

In terms of hardware for VR video capture, cameras have generally been falling into distinct camps. There are sub-$400 consumer-level products like the Ricoh Theta S (offering spherical scene recording in HD at 30 frames per second) and the Kodak SP360 Action Camera (which records at 4K but requires a two-camera setup for spherical recording), along with sub-$600 models like the ICRealtech Allie and Giroptic 360 complementing multi-GoPro setups using mounts like the 360Heros and Freedom360. Then, at the other end of the scale pro-level creators could opt for Nokia’s $60,000 OZO. Now the market is benefiting from better coverage. Samsung’s Beyond has finally materialised in the guise of Gear 360, delivering 360 degree capture in 4K for 350 euros (UK and US prices were still to be confirmed at the time of writing). Then there’s the debut hardware offering from noted software vendor VideoStitch, with its Orah camera launching at $1795, though due to settle at a final retail price of $3,595 a few weeks later.

GoPro has also launched its own mid-level 360-capture solution in the form of Omni. Packing six Hero4 cameras, it’s set to launch somewhere around the $3-5000 mark. It’s professional-level collaboration with Google, the Odyssey/Jump system, which sports a 16-camera array and a price tag of $15,000 still hasn’t arrived a full year after its official announcement. Jaunt One, the much vaunted stereoscopic 360 degree capture device from VR company Jaunt has also yet to see the light of day - a shame given its incredible spec sheet (detailing 60fps, HDR recording at up to 16K resolution per eye.

Interestingly, even Facebook has unveiled its own open source VR pro-level camera, dubbed Surround 360, for the capture of VR footage. Schematics for the kit plus the algorithm necessary to stitch footage from the unit’s 17 camera array (utilising bodies from Point Grey) will be uploaded to Github. Facebook aren’t actually moving into manufacturing and supply business. Weighing in at around £21,000 (making it one of the most affordable VR rigs around) it will be able to hand 4K right up to 8K. Turnarounds of less than a day are promised for the video feed stitching work.

The situation for filmmakers of all stripes is clearly improving, not least with off-the-shelf VR camera rigs eliminating the need to manually synchronise multiple video feeds before editing can begin (though there is still work to done, not least in the area of on-set set gathering, where detailed lens data and rig data, and timecodes to help in the edit remain elusive). Options for stitching, editing and compositing have also dramatically improved. With Kolor now part of the GoPro empire, its Autopano Video Pro 2 tool has become one of the best-known solutions, though VideoStitch Studio also offer comprehensive stitching for a variety of camera rigs and arrays.

AutoPano is available in 299€, 599€ and flavours, while VideoStitch carries a UK price tag just over six hundred pounds. Many existing HDR photo stitching tools can also be utilised for VR video work. PTGui is favoured in many setups, and costs from £78 up to £285 for a three seat licence. In addition to using it for video via batch processing runs, it’s also used in conjunction with After Effects for manual stitching. Microsoft’s ICE and Hugin are two free alternatives to PTGui.

Adobe’s Creative Cloud suite has, not surprisingly, also been a popular option for 360 degree video editing, with a range of plugins launched over the last few months enabling After Effects and Premiere to better handle VR-specific tasks. These include 360VR Toolbox from Dashwood and SkyBox Studio, Fulldome from Multimeios, Trapcode from Red Giant, Campshere from Sat Domes and 360/VR from Mettle. Now Adobe has announced a range of updates for this summer, including a range of tools and improvements for 360-degree/VR work in its key editing software, Premiere Pro. Everyone signed up to a relevent Creative Cloud subscription will automatically receive the updates and additions.

And then there’s Nuke. Thanks to earlier efforts by developer The Foundry to accommodate the needs of users working in stereo 3D, the compositing program is already well placed to deal with VR content. Now The Foundry has just announced a free public beta for its eagerly anticipated CARA VR plugin toolset. CARA VR features stitching tools that help deal with focal lengths, lens distortion and depth-dependant warping, compositing workflows for working in spherical space (including tracking, rig removal and ray-trace rendering) and tools for plate correction and matching. An Oculus-compatible live review process will allow for evaluation directly in NUKE and NUKE Studio.

The catch? While the the Cara VR beta is free-to-use, Nuke remains a professionals-only product, not least thanks to a pricing plan where the cheapest one-seat option still comes in at £2,570, plus £754 annual maintenance costs after the first year.

The necessary tools for VR filmmaking are finally here, but true affordability may take a little longer.