Building your dream workstation: Making sense of storage

Continuing our series on building the workstation you always dreamed of, what do we need to do about storage?

Read part 1, part 2, and part 3 of this series.

Keeping data around while the computer is switched off has been an issue since there has been computers.

Purposes for storage

With modern disks, an array of eight devices can handle demanding uncompressed workflows if need be

Storage for a modern workstation will mainly involve spinning disks and flash memory – both of which are available in various formats to suit different tasks. Storage is mainly used for two specific purposes – for storing the operating system and programs, and for storing video data. Doing these different jobs well requires different things in different configurations and some options might even need plug-in expansion cards to control extra storage.

The operating system and programs are stored on a single disk, called, for complicated historical reasons, the boot disk. It's important that the boot disk can move quickly from one file to another, because tiny files associated with the operating system and the programs may be needed at any time, in quick succession. Also, programs which need temporary space to store data that may be needed quickly, called scratch space, may use the boot disk to do that if there's no alternative. For all these reasons, it's a good idea for the boot disk to be a flash device because solid-state flash disks can retrieve data without waiting for mechanical parts to move around.

This motherboard has six SATA ports. Put a drive on each of them, and a large, fast and reliable storage setup is possible

Video data takes up much more space than programs and since flash is so much more expensive than conventional hard disks, there will almost always be at least some hard disk storage on a 2018 post production workstation. Exactly what's needed varies a lot depending on exactly what sort of work will be done. The biggest setups will store the video in another room on a huge server rack. If that's not what you're doing, the storage needs to go in the computer which will need more space and power. Working with multiple layers of 4K uncompressed raw material is hard work and will need lots of disks working together. A simple one-track edit with HD ProRes is far easier and can be done on a couple. If this is of interest, we could do some more coverage of common codecs from common cameras and the sort of hardware needed.

Places to plug things in

Most motherboards, both small ones for desktop machines and big ones for workstations and servers, have at least some connectivity for hard disks. The most basic type is SATA, an initialism that's longwinded to explain. The original IBM PC AT – for Advanced Technology – used a storage interface called AT Attachment. That grew over decades to become the most common hard disk interface through the 1990s and early 2000s, using a wide ribbon cable with 80 conductors. Pushing speeds ever higher made it difficult to keep all those signals synchronised, so a move was made to a serial bus, which pushes the bits down the wire one after another – hence Serial ATA, or SATA. Phew.

This is a legacy parallel-ATA connector. Most motherboards don't bother including one anymore

Probably the next most common storage interface, by the number of devices sold, is M.2. It's a recent innovation that's designed to provide a place to plug in very fast flash storage in the form of a small circuit board. There's just one M.2 port on each motherboard and it is invariably used to connect a boot device to ensure the computer can start up and handle software quickly.

Really serious storage will be made out of an array of individual devices, which we've covered in detail in the past (see RAID). They'll plug into either a bank of SATA ports on the motherboard or into a plug-in PCIe expansion card. It's perfectly possible to edit reasonably high-quality ProRes or even raw uncompressed video from a single solid state flash disk, but the ratio of capacity to cost usually means that an array of hard disks will be used to store actual video material. Array setups may be SATA, but higher-end setups may use SAS.

Connectors on a SATA disk. The right hand, smaller connector is for data – the longer left hand one is for power. Modern power supplies will have SATA power connectors

High-end disks

SAS, standing for Serial-Attached SCSI, replaces the older SCSI interface in the same way that SATA replaces ATA. SAS disks are likely to be, in the words of the industry, “enterprise-grade,” with various aspects of their engineering upgraded over SATA equivalents. That also means they'll be expensive, especially in multiples, so that even quite high-end workstations may use SATA disks. RAID controllers built for SAS disks can use SATA disks, via suitable adaptor cables, as a cheaper option. Many motherboards include RAID provisions for SATA disks which are actually very usable for basic video work. Plug-in cards may offer more, with a lot of variability in terms of price and features.

Both of these are hard disks. Flash SSDs are often the same size and shape as the smaller drive. All of them use the same data and power connectors – the system doesn't care

Long-term storage

Neither flash nor hard disks are a long-term archival format. This will be a brief mention as the format is too expensive for many people, but many video workstations would benefit from an LTO drive for backups and archiving. All LTO drives are, internally, SAS devices and will need a SAS expansion card to plug into. LTO drives advertised as being USB3-compatible are SAS drives in a box with a USB3 converter added. LTO drives are expensive (thousands each, at the time of writing) and the tapes aren't exactly pocket money, but there are very few other options which provide the same sort of performance and (estimated) permanence.

Next time, to complete the core parts of a modern workstation we'll look at GPUs, which have recently become as crucial as the CPU to the performance of a post production system. In the meantime, leave any queries in the comments below, because storage is a huge subject.