Whatever happened to the flash revolution?

A Raid array is still at the centre of most facilities

While flash media is an increasingly ubiquitous storage solution for consumer devices, thanks to its ability to stay ahead in terms of the space-to-money ratio much of video production still relies on metal spinning disks.

It's been nearly ten years since solid state disks started, oh so gradually, to become a realistic alternative to spinning metal hard drives. Despite rapid and significant improvements in price-performance, though, flash has still not quite overtaken traditional hard disks in markets where sheer space is important, which is perhaps slightly surprising. The hard disk companies, it seems, have been ably keeping up.

Flash forward?

The advantages and disadvantages of flash hardly bear repeating. Compared to hard disks, a flash drive is smaller, lighter, quieter, consumes less power, produces less heat and is faster, particularly at things like random reads (meaning reading bits of files which may not be right next to one another on a physical disk). A flash device, of course, has no meaningful concept of 'right next to one another'. It's just a list of numbered cells, any of which can be accessed at any time. All of these  are reasons why flash storage is in the active process of taking over data centres. These data centres are the places where your data actually goes if you send it into 'the cloud', which isn't actually a fluffy white thing in the sky. The cloud is a big warehouse full of computers on a nearby land mass where space, talent and power are cheaper.

Data centres like that are very sensitive to physical space, heat and power consumption. Depending on the work they're doing, the ability to grab tiny files from all over the drive is crucial. Web serving, for instance, involves lots of random reads, as all of the tiny images and fragments of text which form a website are built or retrieved. With the current model of machines on demand, even if a machine in a data centre isn't doing that sort of work right now, it might be in five minutes.

Portable recorders take advantage of the size, weight, power consumption and performance advantages of flash.

And that's just one application. The size and power benefits of flash are a no-brainer for laptops or for desktop and media-centre machines (or even games consoles and set-top boxes), which are expected to be small and light, start up quickly, consume little power and make little noise. So, in a lot of circumstances, moving entirely to flash storage makes absolute sense, both financially and from an engineering perspective. The one place in which it still doesn't, sometimes, is the video workstation, where the use case is, to put it succinctly, weird.

Naturally, we're all very used to flash as portable media, for cameras and recorders, where many of those advantages that matter to laptops also matter to us. On the desktop, though, we're only using one machine, so we often don't care so much about power consumption as people do when they're building server farms. We do care about speed. If we're cutting uncompressed 16-bit 4K at 60 frames per second in stereo 3D, we probably care quite a lot about speed, but at over six gigabytes per second, we very soon have to start caring about space. These numbers will become embarrassingly out of date very quickly, but at the time of writing, Crucial's MX300 SSD, of two-terabyte capacity, costs nearly £500. (Hello, people reading this in 2019, or possibly March next year. We can hear your uncontrolled mirth from here.)

Video Devices' clever approach is to support commodity M.2 drives in a USB3 caddy,
keeping costs down and performance up.

Regardless, a Seagate Barracuda 7200.12 two-terabyte hard disk is bigger, heavier, noisier, hotter, consumes more power and only goes at about 160 megabytes per second, but it only costs £65.99. Flash storage has collapsed in price since its introduction, exactly as we all hoped it would, but it's still lagging massively behind on space-to-money ratio. Spinning metal has been able to keep up the pace such that, for the last few bi-yearly workstation upgrades, maintaining at least the same amount of space and speed has continued to be much more cost-effective on disks rather than flash.

OK, fine, users would need more of the Barracudas than MX300s to achieve the same speed. Compounding matters, if we are in a situation where we really need a lot of disks, then the cost of a controller card becomes significant.

It's worth a quick recap of our piece on RAID techniques. Buying very expensive controllers capable of RAID-5 or RAID-6 in order to reduce the number of required disks can be a false economy against simpler, faster approaches like RAID-10, because more disks are cheaper than big clever controllers, if you aren't too fussed about power consumption. Often, modern computers can host enough disks to do a perfectly workable job using inbuilt controllers.

Upscale storage like Quantum's StorNext, away from the desktop, is a hard disk holdout, too.

Storage stragglers

So, much as video production was the last holdout of tape and, to an extent, continues to be via LTO, it also looks like video production will likely to be the last holdout of the desktop hard disk. Really big storage, as used by news channels to store massive quantities of material, is another case where sheer space is the deciding factor, but that's usually hidden in the basement. Perhaps at some point, the innovations of recent hard disk history (perpendicular recording and so on) will run out and flash will catch up, in terms of price per megabyte.

The question then would be whether we'd have gained faster flash or whether we'd have lost ever-bigger hard disks.

Main image: www.shutterstock.com