Much faster and better storage is arriving, and it will totally change the way we think about codecs

Wouldn't it be nice if we could forget all this complexity over codecs and compression and just store video material in universally-recognised formats, like sound people do?

What's stopping moving picture people doing that is, of course, storage and the need to compress. As we've seen recently, though, flash storage is a huge market that attracts massive investment from a much wider base than the film industry. That investment might soon make it possible to store uncompressed, high-resolution video without having to worry too much about the practicalities.

Recent advances in several areas of electronics have involved stacking microchips one on top of the other. Most flash these days is three-dimensional to some extent, with dozens of layers of silicon wafers inside each plastic package. This somewhat mirrors developments in sensor tech, with the ability to stack up an ideal photosensitive layer on top of a ideal electronics layers, without having to compromise any of them to make the others possible. That's not quite the trick with flash, of course. Flash is just about fitting more bits in every finished, packaged microchip by stacking 'em high and, with luck, selling 'em cheap.

Toshiba

A recent incarnation of this was shown at CES last month, where Toshiba had a flash chip a full ninety-six layers deep called the BG4, for its ball grid array package. As far as most of the world's press was concerned, BG4 is important because it reduces the thickness for a given capacity from 1.5 to 1.3mm. Perhaps camera people aren't so worried by fractions of a millimetre, but Toshiba will have an eye on the market for compact laptops and servers designed to pack densely into facilities which are charged by the rack unit. In the world of film and TV, many cameras now use commodity flash, whether or not that commodity flash has been rehoused in a nice milled aluminium enclosure. Toshiba's new device might be more or less immediately deployable, especially as there's both NVMe and M.2 versions, the latter of which would slide into at least some cameras without modification.

The BG4 is a QLC device, which notionally stands for “quad-level cell.” These descriptions are a little counterintuitive because a QLC flash chip actually stores four bits in each cell, which makes each cell capable of storing sixteen discrete states. Multi-level cell flash was traditionally avoided for high performance applications (and storing uncompressed video at, say, 8K is a high performance application.) That seems to have been overcome, though, because Toshiba claim the new device is even faster than the previous BG3 device, which achieves three bits per cell (yes, we call that TLC, though it's really better thought of as eight states per cell.)

So, where does this get us?

Let's do the sums. A full 8K frame hovers around 38 megapixels, assuming we make our frame 8192 pixels wide, as opposed to multiplying up an HD frame to get to 7680. If we want to store that 8192 by 4680 raw Bayer mosaic at 16 bits (or two bytes) per photosite, we're talking about something like 72MB a frame. Assume we want to achieve something like 60 frames per second. That'll demand about 8192 × 4680 × 60 × 2 = 4387.5 = 4.4GB per second of storage.

There wasn't really any single widely-available flash storage device which would do that before Toshiba's announcement, and, sure enough, there still isn't now. The new device claims a 1.7GB per second write speed; specifications invariably go on to discuss input/output operations per second, IOPS, which are based on agreed industry metrics and key to things like file servers. The workload presented by high bandwidth video recording isn't very well represented by IOPS numbers, and these statistics aren't very useful to us. The BG4 device is both very fast – fast enough to make even pretty serious 4K workloads trivial – and not quite fast enough to make 8K raw an everyday occurrence.

The thing is, it's clearly going to happen at some point. Flash is expanding so fast that the longed-for future of simple, patent-free, cross-platform raw is inevitable – at least so long as we don't keep quadrupling the frame sizes produced by cameras every few years.