How to test cameras part three: test them in the real world!

In our article How should we test cameras? we asked what was better: unblinking scientific testing or judging on the percieved quality of the image, and we discussed some of the issues surrounding camera evaluations. But we thought it was only right to get the perspective from the people on the front-end doing the reviews themselves. Here, Roland Denning says the ultimate test takes time.

My answer is simple - shoot a film on one. Every test reel now looks great - we’ve seen all those shots of the city at night, the docks at dawn, the bustling market, the sunsets... do they actually tell us anything?

We all agree there are certain objective criteria that can be quantified — dynamic range, sensitivity, resolution, sensor size — but we are getting to the stage when all professional cameras can turn out great pictures. What we need to know is what the cameras are like in the real world, when conditions are far from ideal.

Even the quantifiable aspects have subjective elements to them — one camera may claim higher sensitivity but may be no better than a lower spec camera with the gain turned up. When does picture noise become unacceptable? And whatever the quoted dynamic range, what do the pictures actually look like in high contrast situations? Does the viewfinder give you any idea of what the pictures really look like?

There are also the basic ergonomics of the camera: how does it balance on the shoulder, can you operate in cramped conditions, are there controls that keep getting jogged because they are in the wrong place? How many menu steps do you have to go through to for the most common adjustments? Are any of the picture pre-sets of any use at all? Do you really need a DIT to get the best out of this camera? Some judgements may be just to do with familiarity - camera design is fundamentally conservative, and if a manufacturer tries to do something in a different way, initial reactions might be ‘what the hell is going on here?’

Often you have to live with a camera for days or even weeks to answer these questions.

Take, as an extreme example, the BlackMagic Pocket Cinema Camera. A pocket-size camera with an amazing spec at a bargain price, but soon you find that to use it in many real-world situations you might start adding enough accessories — rigs, external batteries, monitor, audio recorder — that mean a) it is no longer pocket sized and b) it is no longer quite so cheap. Actually I own one of these and I think it is great little camera but it took me a while to actually work out what it’s best for. It’s fantastic in certain situations, useless in others, but you need to live with it find out what those uses are.

However hard you try, it is difficult to make a camera test difficult for yourself but, in the real world you are forced to make compromises: the light changes suddenly, the room is too small, there is no space in the back of the car, the subject is running away, the battery has failed with no warning... that’s when we find out just whether this relationship is going to last.

Cinematography in the end is not about making beautiful pictures or capturing reality but telling stories; cameras and the all the rest of the equipment we use are just a means to that end.