It’s been a summer of superb innovation from the stunningly talented group that call themselves Magic Lantern. Not content with turning Canon EOS cameras in to RAW video monsters, their latest innovation is breathtaking: an extra 3 stops of dynamic range. Phil Rhodes explains how they’ve done it
Magic Lantern's latest innovation is being quite widely discussed at the moment, and is certainly a very impressive bit of reverse-engineering. The difficulty associated with reprogramming something as complicated as a modern DSLR without the support of the manufacturer shouldn't be overlooked, and to date they've provided a wide variety of interesting and useful tweaks.
Up until now, however, Magic Lantern hasn't got right down to actually tweaking quite such fundamental imaging parameters. The claim is that the 5D Mk. III is now capable of producing images with 14 stops of dynamic range. It's widely accepted – and confirmed from back when I first tested the Mk. II – that Canon's current electronics are capable of very slightly less than 11 stops natively, so this could be a pretty significant feat. Since as every stop represents a doubling of intensity, the implication is that the brightest resolvable highlights are now up to eight times brighter than once they were.
The technique behind achieving this is connected to the internal architecture of certain cameras in the range. It's pretty normal for CMOS sensors to have more than one channel through which image information can be offloaded, dividing down the amount of data per channel and alleviating the issues associated with sending signals down wires very, very quickly. This is the case for sensors which offload both analogue and digital information, the difference being whether the analogue-to-digital converter that turn the output of the light-sensitive pixels into digital information are on the sensor itself or not.
All of the Canon DSLRs we're aware of use a separate device to digitise the sensor signals; a custom component made by Analog Devices, a major manufacturer of high-speed A/D converters (their parts are on a lot of Blackmagic's products too). These devices are used to do various jobs, not least of which is the amplification required to alter the camera's sensitivity in response to ISO speeds selected by the user. Since at least the 5D Mk. III and 7D use several channels of communication to offload the sensor, there are several sets of amplifiers on these components. Crucially, they can be individually controlled.
What this means in practise is that it's possible to make these cameras read out alternate pairs of rows from the sensor with different amounts of gain. I say pairs of rows because that's simply how the cameras are configured, but also because that's how you would actually want it – if you took single rows from a Bayer sensor, you wouldn't get a red, green and blue photosite that were anywhere physically near one another, which would cause enormous problems in reconstructing a full colour image – and reconstructing that image is where we start to encounter a few compromises.
The data that's recorded by cameras using this technique is of course not directly viewable using normal software; the requirements for reconstructing a reasonable, viewable image from data that is not only in a Bayer mosaic but also contains alternate stripes of variable exposure information is reasonably complicated and beyond the scope of an article like this. Even so, it's intuitive that there are compromises involved, just as there are compromises in using Bayer patterned sensors to begin with. These compromises are visible in finished images as increased moire and aliasing due to the lower effective fill-factor of the sensor (that is, big gaps between the effective pixels). This is especially true in video mode where the sensor is already skipping lines or binning pixels in order to achieve video frame rates.
The practical effect of the fill-factor problem is that where both exposures both contain useful information, the image can look very good, subject to the development of reasonable reconstruction algorithms. Where only one of them contains useful information – where the brighter exposure is clipped to white, or where the darker one has fallen off into noise – the vertical resolution is halved, since that area of the image must be reconstructed from only one of the two exposures, which contains only half the vertical number of pixels.
Either way, this new technique works for both photo and video modes on the 5D Mk. III (and for stills only on the 7D, which has no raw video option at the moment). Perhaps the most excitement comes from people interested in HDR video, who are to an extent used to dealing with a certain amount of aliasing. Footage available online suggests that the degree of aliasing can be really quite severe using current reconstruction techniques, especially in video mode where there's already a certain amount of line-skipping going on. Whether this aliasing and moiré is more or less objectionable than the noise required to achieve equivalent dynamic range using conventional settings is a matter of taste. Certainly, being immune to the problems moving subjects, it is much more convincing on them than, say, Red's HDRx or previous sequential-exposure attempts at HDR from Magic Lantern themselves, at least when you're using your moving-picture camera to photograph subjects that are actually moving.
Whether this new capability actually does what it says is currently somewhat open to debate. As with any electronic camera, neither the ISO programming nor techniques such as the one we've been discussing can actually alter the performance of the sensor. The sensor – unless there's something extremely exotic going on – is a fixed piece of silicon which does what it does; selecting different ISOs simply alters the amplification of the signal coming out of it. That being the case, we are not actually taking an “underexposed” and “overexposed” image and combining them, as we are with conventional HDR techniques which use different exposure times for each shot. With this new technique, both of the images arise from sensor data in which the exposure characteristics are identical. A “bright” exposure, treated as ISO3200 then attenuated down to form the shadow detail of a “dim” ISO 100 exposure, has effectively been amplified then un-amplified again. Quite why this Machiavellian piece of signal processing produces quieter blacks is a question for the people who designed the 5D3's signal chain, but it may be a simple matter of amplifier dynamic range limits or a gain-structure decision aimed at creating equal dynamic range at different effective ISOs. There might even really be something extremely exotic going on.
Regardless, one might reasonably ask why Canon didn't simply build a better signal chain, so that the camera could read all of this information in one pass, and avoid the compromises we've discussed. After all, if simply modifying amplifier gains can bring out more information, it must be on the sensor to begin with, right? Well, strictly speaking, yes, but that's where you get into the level of engineering that costs tens of thousands of dollars a camera. People used to the sorts of cameras that cost that much might dismiss Magic Lantern's new HDR hack as lacking in real substance, but it's hard to be that dismissive of something that already works astonishingly well, costs nothing, and is doing some genuinely interesting engineering, especially given the circumstances they're working under.
The bottom line is that it does appear to work, even if the noise suppressed by this approach is noise that was either introduced or made more visible by the way the signal chain handled the information from the sensor, as opposed to actually making the sensor less noisy. Probably the reconstruction techniques will get better over time, but there's always going to be a downside. Whether you want to use it is, as with DSLRs themselves, down to whether you can put up with the caveats.