Products like the Blackmagic Cinema Camera and the upcoming Digital Bolex D16 - not to mention Magic Lantern-hacked EOS cameras are all capable of producing raw video. Just a few years back, virtually no-one had encountered this strange new format. Now, it's all over the place. Peter Haas looks at how you deal with this stuff in the real world
The first part of this article, Preparing to shoot raw, is here
Previously in Preparing To Shoot Raw, we discussed the potential storage nightmare that is involved with working with data-munching formats such as 2k, 4K, and various resolutions of raw. The math shows that even with cost of storage falling, the cost-per-minute of safely archiving and creating edit-friendly media adds up quickly. As many readers pointed out, this article was based around working in what we refer to as an "online" workflow. So here, by popular demand, are the basics about working in an "offline" situation, why it works, and how it can save you money.
The overall "offline" (proxy/workprint) workflow is relatively simple and goes something like this: you import your footage at a lower resolution which takes up less hard drive space and is less strenuous on your computer, edit with that media and when you've completed your project you re-import just the footage on your timeline as full resolution and erase the workprint files.
This workflow isn't anything new, it comes from the days when hard drive sizes were small and extremely expensive and processing power was low. Studios needed to squeeze as much mileage out of every megabyte as possible. Then as computers and media storage got better, the industry slowly weaned itself from this workflow. Now with the advent of raw and 4K, we are once again in a place where the demands of working with digital footage are greater than the processing power available to most editors and filmmakers.
This is the reason I have such a problem with the tech-speak of digital video & cinema. "Offline Media" refers to confusingly two completely different things. In one instance, you have the dreaded "Media Offline" red or blue title card warning you that your footage has gone missing. This is typically, very, very bad. In the situation we'll be discussing here, "offline media" refers to low-resolution (read: smaller file size), copies of your footage containing matching meta-data.
Notice that I included the words "with matching meta-data". The only way that this workflow can operate smoothly is if the editing software has enough information to match the offline media back the original media to create new high-resolution copies.
Each card/tape you shoot should be named as its own reel. Each reel contains timecode, and timecode indexes each frame. Think of it like referencing a book. REEL is like the name of a book you want to read from, and timecode is like telling the software which "page" to turn to.
With this information you can even change the frame size of your project, say ingesting a 4K project at 2K proxy, saving even more precious hard drive space. No matter how you change the dimensions of your picture, as long as the timecode and reel names are locked in place, the computer will always be able to find and match back to the original material.
This is why camera reel/tape names and timecode are such an important part of professional workflows. If your footage does not have this information, you will not be able to use advanced workflows.
Note: Most professional tapeless cameras allow you set reel and timecode information. If you're working with a camera that doesn't, I highly recommend you check out Eyeframe Converter and Video Tool Shed QTChange. Both are software that allows you to add both Reel and Timecode manually.
Importing the footage at a lower resolution is simple, and is done when importing into the NLE*. Every software works a little different, but typically you can tell your NLE to bring the footage in at a lower quality resolution in the project or import settings. The recommended formats for an offline edit are Apple ProRes Proxy, Avid DNxHD36 or PhotoJPEG. There are other formats out there, but these are the ones I find easiest to work with.
After you've completed your edit, you tell your NLE or finishing software to look back over the camera reel and timecode information, creating full quality footage only for the pieces that are on your timeline.
Again, every NLE is a little different, but the core concepts of working in an offline environment will be the same.
Let's take a look at the feature film project from the previous article.
We created a hypothetical narrative feature, shot at a ratio around 15:1, which the camera negative yielded 8.3 TB (yes, with a "T"!) of information for one copy. This quickly expanded to 25TB once you've created your 2 extra back-up copies. Offline editing can't help here. You need this archive copies to keep your project safe. I wouldn't be doing anyone a service by recommending otherwise.
Where you'll start to see the saving is in the edit.
Let's assume you're going to do an edit in Apple ProRes Proxy for editing. This proxy format runs at 19GB/HR versus ProRes HQ which eats up 91GB/HR.
(Again, to reiterate: I’m not a math-whiz, and a majority of these numbers are rounded for the sake of simplicity and clarity BUT they do realistically represent the data we are dealing with and have served me well in the past).
| Format | GB/Hour | Hours | Total |
| ProRes Proxy | 19 | 30 | 570 |
| ProRes HQ | 91 | 30 | 3 TB |
You can see a massive drop in hardware costs here. Being able to fit your entire project on less than 600GB worth of space opens up a lot of resources.
Working with raw footage has it's own unique set challenges. Most of, if not all professional NLE software lacks any support for directly importing these files in any manageable format. This means you have to turn to a third-party software solution to create an edit-friendly files. So in away, EVERY project you do with raw is in some way or another "offline".
If you are looking to edit raw files with proxies you need to make sure that the raw processing software you use maintains the every-important meta-data. Without this, you'll be lost in your online.
As of this writing, the most complete RAW workflow software I've been able to find has been Black Magic Resolve (though I'm excited to see what Pomfort's ClipHouse has to offer once it's out of beta!). I've only used the free edition, which means I'm limited to HD output, even from 2K files. What makes Resolve such a strong product is the ability to completely round-trip your raw footage from the "camera master" to your editing software and back to Resolve for finishing.
Resolves ability to stand-in for the media management tool is essential in dealing with raw footage. Since your NLE is not able to import/transcode raw footage, your ability to match-back proxies to the master media would involve a lot of manual transcoding of files, and there is nothing worse than having to go through your edit, shot by shot, frame by frame to figure out what you have to transcode again.
In Resolve you create a project to transcode your raw footage into proxy formats. When you've completed editing you can export an XML or EDL which allows you to bring your edit back into Resolve and match-back to the raw camera masters. This allows you to do one of two things: export a new set of higher resolution media to complete your online with, or, if you have the processing power, complete your picture online with the awesome flexibility of the raw files.
As the raw file format begins to gain more popularity, it will be interesting to see when and how editing platforms begin to adopt the ability to treat raw camera masters like any other camera reel.
Sure, working offline saves you from buying a few extra hard drives, but the real savings comes from less directly measurable places.
First off, it saves you from having to constantly upgrade your equipment with every shift in technology.
Buying into technology is always difficult. You know as soon as you lay down your money that something faster and sexier is just around the bend, and you live in fear that the next upgrade to your favorite software is going to require a generation of hardware unavailable to your setup (Being an Apple user since the IIe, I've felt this sting all too often).
My point here is that just because you were hired to produce your next sitcom in 3D doesn't mean you should have to scrap your old post stations and blow a third of the show's budget on new gear (this happens more than you think!). The flexibility of an offline workflow allows you create proxy media in the format that is best suited for your current editing environment and resources. This means you'll be able to continue using your current equipment for longer periods of time, thus extending the return on your investment.
More importantly, you're going to be saving you or your editor a huge headache during the edit itself. Your NLE will be more responsive to proxy / offline footage simply because the processing power of your computer is far greater than what is necessary to playback and scrub through it at high speeds. With feature length projects fitting on tiny portable drives, your editor can even cut on a laptop from the beach. And isn't keeping your editor happy the most important thing?
Happy cutting!
The first part of this article, Preparing to shoot raw, is here