Video data is doubling every few years. How do you keep up with it? [sponsored]

As we can have more data, we use more data, and it’s not just the impact of higher resolution formats on video production that mean we have to learn how to wrangle it in smarter ways.

Amidst all the debate about new resolutions and new formats, one thing is certain: every year, we need a bit more data to accommodate it all. Whether as filmmakers in the way we make our living or just as human beings out and about in an increasingly connected society, our data requirements are growing all the time.

We tend to fix on the front end of all this. We look at the pixel count of the resolution or the model number of the smartphone or the iteration of the OS, while the number of gigabytes that are associated with this progress is fading into the background. That’s because we assume it will always serve our needs. But that is not always the case.

The iPhone is as good an example of this as any. We’ve tracked iPhone development on RedShark from the outset and watched it grow into an indispensable shooting aid, even the capture tool itself on occasion. Once upon a time it would have been hard to imagine that a phone would need 64GB just to function reliably in a modern data environment. A 64GB iPhone didn’t even exist until the iPhone 5 in 2012. Now it’s the entry level spec and teetering on the verge of being obsolete.

Chasing storage capacity

The problem is that the amount of storage creatives need is usually very close to the amount of storage we have at our disposal. Give software developers a 64GB bucket and they will fill it fast enough that pretty soon you’ll be looking for a 128GB one. And with film and video the problem is even worse. It’s not just a case of needing more storage for larger file sizes as we transition from HD to 4K, the way we shoot is changing as well.

Famously, Alfred Hitchcock’s shooting ratio — the amount of footage required over and above the amount used in the final film — was a mere 3:1 thanks to meticulous storyboarding beforehand. Hitchcock’s primary driver for this workflow was the desire not to have his films changed and re-edited on the studio’s say so: given that small amount of material there was only one realistic way that the film could be cut together to make an overall narrative

This is very much an outlier and takes an inordinate amount of discipline to pull off. Typical Golden Age Hollywood films budgeted a 10:1 shooting ratio, the same as a modern low budget movie. Meanwhile the background level throughout the industry is probably closer to 20:1.

That, though is film. Digital acquisition has not just blown those figures out of the ballpark, it’s knocked them into another stadium and a whole new game with very different rules.

High shooting ratios

The shooting ratio has skyrocketed over the past couple of decades, with even that 20:1 figure under pressure. The move towards digital has meant that the fierce discipline of only rolling during the parts between ‘Action’ and ‘Cut’ has become a hangover from the past: action as likely to mean pointing the camera in the right place as much as pressing record. Cameras now shoot for extended periods to capture anything that might be useful, from unexpected set-ups and bits of improv, to extra material for promos and more.

Again, there’s an outlier here, and it’s an impressive one. Mad Max: Fury Road’s shooting ratio was 240:1. That’s 480 hours of raw footage shot for a total running time of 120 minutes.

Of course, few productions are ever going to scale those dizzying heights, but you cannot escape the fact that you talk to anyone in the editing room and they’ll say they have to deal with more material every year. It’s not just because people are doing more takes either. The increasing use of two-camera set-ups means more data. VFX shots mean more data. Versioning for different formats means more data.

The simple fact is that if you’re making any video production, you’re going to need more storage. Even as steadily more advanced codecs try and chip away at the overall amount, 4K data is going to take up roughly three times the storage volume of HD data. 360-degree video will take up more again. 8K? Near triple again.

Meanwhile we have become used to having lots of data; technology has almost become profligate with it. The way we work and the way we live depends on being able to carry vast reservoirs of it around with us and be able to tap into them when required. Every second, a million minutes of video content will cross the global IP network by 2021. Producing that and storing that is going to require that you have the best storage available to keep up.

For more information on the d2 click here (UK) and for more information on the Big RAID storage click here (UK). For those outside of the UK please head to www.lacie.com for more information.