How we capture images in the future will be like nothing we can imagine

The Lytro Cinema Camera shows us that the future of cameras is not set

Inasmuch as anything's predictable, we might reasonably predict that cameras in a few years time will look absolutely nothing like cameras of today.

The fundamentals of photography haven't changed in nearly a century. A lens focuses an image on something that will react to it. There's only been a handful of technologies capable of doing that and we've never moved beyond glass lenses. The performance of silicon-based imaging sensors is approaching theoretical limits on things like sensitivity and dynamic range. At the same time, we're starting to see resolution performance which leads us to question whether more is actually useful. And finally, there's the need of manufacturers to release something new every few months.

These three factors combine to suggest that the next five to ten years may bring quite significant changes. They each drive change in a different way. Limits on sensitivity and dynamic range are a target for workarounds, so R&D people will endeavour to do so. We can make sensors smothered in more pixels than we need for sheer resolution's sake, so perhaps we'll start doing other things with those pixels than just directly sampling the scene. And finally, yes, there'll be money to do these things because the people who sell them are as keenly aware as the rest of us that the endgame of traditional cameras is possibly nigh.

Light-field cameras

A hint at the future is represented by the light-field arrays Fraunhofer demonstrated (who would seem a reasonable source of new innovations.) Right now, if we take a four-by-four array of small cameras, we can have the computer work out novel camera positions anywhere within that square area and we can use the three-dimensional information made available by the array to very accurately simulate optical effects such as depth of field. Since more than one camera can see most areas of the scene, we can add all of those pixels together to achieve higher dynamic range and/or lower noise.

We could also do that with a larger single sensor, but the smaller cameras have a large depth of field so we fight less with issues of soft focus and we can simulate this later, very accurately, if we'd like to. There are few drawbacks if we can get the computer software right, though there are fundamental problems there, too. If one camera can see two hairs side-by-side on the head of an actor, there's no completely predictable answer as to which of them is visible to an adjacent camera which can only see the one that's in front. These are are some pretty low-level issues that make perfection hard to achieve.

Most of the light-field-like setups which exist are a little ungainly, a bit heavy, but they hint at the future, and the future probably isn't one in which a single piece of silicon nestles behind a single conventional lens. If we imagine an even bigger light-field array — perhaps a spherical one, looking inward, the size of a sound stage — we'd have a studio in which all the action could be performed and shots composed after the fact. That would involve a number and density of cameras, an avalanche of data and a post-processing workload that would significantly exceed the practical capability of modern hardware, but there's nothing that makes it theoretically impossible.

The Lytro Immerge isn't exactly small, but it could be a glimpse into the future of studio camera setups - Lytro

The artistic point of view

There are some things that make it undesirable, though, and they're not technological. They're artistic, because film and TV is an art-form of now nearly a century's seniority, and people like certain things about it. We don't just perform the action and point cameras at it. We adjust the action to make the cinematic presentation of it more to our liking. Nobody's going to walk onto our theoretical stage, wallpaper it with a gigantic light-field array and start directing actors. They're going to want a motion tracked, wooden camera mock-up — a security camera, in the sense of security blanket — with a real-time feed of the virtual camera as reconstructed at that location, just like motion capture productions do. So, our light-field studio doesn't only need to be able to record all the data, it needs to be able to process it in real-time.

We're heading wildly off into speculation here. None of this has even been seriously proposed and it all begs the question as to whether it's worth so carefully simulating the behaviour of a monolithic camera when we could simply use a monolithic camera. What does seem likely is that cameras will stop being the monolithic blocks we have today, at least in some circumstances, perhaps becoming a property of the venue rather than a piece of equipment. In other situations, people will want to keep being photographers in much the same way oil painters still use archaic tools and materials. Still, this is exactly the sort of thing that commercial cinema, the stuff that needs to be produced to order and to fulfil a certain very commercial set of very commercial requirements, might latch onto very enthusiastically.

There are other things we might see done. We've talked about light-field arrays (or more accurately, simulated light-field arrays) here because they've been shown, they work and they're fairly well-known. They help out with camera technology but also with things like lenses, where they can simulate things like anamorphic focus behaviour. Other than light-field simulation, there isn't really any grand new development on the horizon. Neither is there anything much going on in audio, which we might have expected by now to be recording a complete phase-accurate map of the pressure waves in the air so that any arbitrarily directional microphone could be simulated — a phased-array microphone.

Whether or not any of this speculation turns out to be inaccurate remains to be seen, but it seems pretty safe to return to our idea that cameras are probably about to change significantly for the first time in more than 100 years.