108mp in a smart phone? There's logic in the madness

Putting 108mp onto a smartphone sensor might not seem, well, very smart. With rumours abound about the Samsung Galaxy S11, we examine what on earth the company could be thinking, and discover that there could well be some solid logic behind it.

At the time of writing, the world, or at least that part of the world which gets excited about smartphones, was breathless with anticipation over the upcoming Samsung Galaxy S11, which reportedly has a 108-megapixel camera and is capable of 8K video recording. Great as smartphone technology has become in the last few years, announcements like this are probably best taken quite carefully, if only because the underlying technology is probably, genuinely quite clever, and likely deserves more sober attention than the headlines would imply.

The first thing to make clear is that all of this is based on information that may have carefully been deposited in the ears of journalists by Samsung in order to create a buzz, or may have been smuggled out of a top-secret design lab by a laser beam-dodging super-spy in a black catsuit, or may have been made up using a random text generator by someone eager to enhance a reputation as a fount of all knowledge. Even if accurate, these pieces of information will at the very least be the top few picks from the advertising department’s notebook of fun facts about the product.

Sharp eyes (ha!) will already have noticed one problem with the two specifications we’ve mentioned. If it records 8K video, it probably doesn’t need a 108-megapixel camera, and reading around the subject reveals references to pixel binning. That’s fine; averaging pixels together is a neat illustration of the ways in which we can trade off resolution for noise performance – or, if we consider the edges of pixels to be noise, which we probably should, trade one kind of noise for another. Divide 108 by four, given a 2-by-2 pixel average that we’ll discuss below, and we end up at 27 megapixels, which is actually a little less than the 33-megapixel resolution of an 8K video file.

So, the devil is in the details, but it’ll have a reasonable claim on the term “8K video.” What might be even more amusing is if two or more of those cameras are to be used for depth capture, effectively forming a very small lightfield array. Two has been done before (that’s just a stereo pair) but more is rare; we’ve talked about Fraunhofer’s lightfield technology before, and gone into some detail on how it works and the very impressive things it can do. One day, a phone manufacturer will put a camera in each corner of the phone, and rather abruptly it’ll become easier than ever before to extract depth information from the resulting image. If there’s a pressing public demand for that sort of feature perhaps it’ll happen one day; quite a lot of effort has already been put into features which approximate that sort of feature and use it for tricks such as simulated depth of field.

How meaningful is 108mp?

Of course there are all of the usual questions about how meaningful it is to put 108 million pixels on a sensor the size of one we’d typically find in a cellphone. Sources tell us that the ISOCELL HMX (for that is its name) is a 1/1.33-inch, or roughly three-quarter-inch sensor, which is large for a cellphone. Even so, 108 megapixels would be hugely ambitious number for a full-frame DSLR, and the tradeoffs of noise against photosite size are well known. Make the light-sensitive area smaller and photons are less likely to hit it, so the signal, for any given scene, gets considerably smaller and needs more amplification.

Samsung calls its response to this reality “tetracell,” which seems to imply that each of the red, green and blue-filtered photosites in each Bayer cluster actually involves a cluster of four photosites which can be averaged in low light to trade off resolution for sensitivity, or used individually under better illumination. Averaged together, the total size is a little larger than the photosites in an iPhone XS (1.6 versus 1.4 micrometres). Like many phones, it’s likely to be a good camera (for a phone) in good light, but the selective averaging approach is smart; this is the real purpose of the high photosite count.

The Galaxy S11 is apparently to include a variety of other features, including a 120Hz display (which will at least make it possible to display slower frame rate video more accurately). Anyway, the march of the cellphones continues unabated. It seems to be a field curiously immune to the situation that pertains with – say – audio gear, which long ago reached a level of sophistication that was more than adequate for its purpose while simultaneously being very affordable. Professional cameras are, arguably, approaching the same zenith. 120Hz is the point at which many people agree that further increases in frame rate are no longer visible to anyone. Likewise, 8K is, finally, closing the book on the resolution argument. Whether that’ll stop the cellphone people going even further remains to be seen, but it’ll be fun finding out.