Nikon's combined rolling shutter/global shutter sensor patent revealed

Global and rolling shutter sensor designs both have different advantages and disadvantages. So, why have one or the other when you can simply have both on one chip? That's exactly what Nikon has proposed in an updated patent, but with a twist.

It has been the long held view among internet commentators that the ultimate endgame is for all shutters to be of the global variety. For those of you who are less technically inclined, CMOS sensors come in two flavours; rolling shutter and global shutter.

The difference between them is that in the case of a rolling shutter design, the sensor is exposed to light line by line. On most video cameras this is performed top to bottom, although a CMOS sensor can also be read out in this way in the reverse direction. The unique ability of a CMOS sensor to do this was one of the main reasons why the original 1/2" Sony EX series cameras were possible in such a compact body, since the three CMOS sensors contained within the camera could be mounted more conveniently with less optics necessary than a traditional three-chip design.

The big drawback with exposing and reading out each line one-by-one is that if the camera performs a fast panning motion, or a moving object such as a train moves horizontally across the frame, it results in a unnatural looking skewing effect. On older cameras, the readout was so slow that if you moved the camera quickly side to side, the image would appear to wobble like jelly. On modern cameras the readout speeds have been increased to such a degree that the skewing effect is mitigated to a large degree on all but the fastest moving objects or camera moves. But, it remains a drawback of the design nevertheless.

A global shutter design, on the other hand, exposes and reads out the entire image in one go, so the skewing effect is non existent. It's easy to see why a global shutter is desirable, particularly given that previous to CMOS sensors, CCD sensors were used exclusively and had a more 'global' style readout by reading each column of pixels simultaneously.

The trouble is that global shutter CMOS sensors require more electronics on the chip, which creates noise and more heat. The increased heat can affect the signal to noise ratio even more, and dynamic range is also reduced compared to an equivalent rolling shutter sensor. Additionally, the increased number of transistors on each light capturing photosite (you can also think of a photosite as being the equivalent of a pixel) can also lead to a smaller pixel pitch, since there's less room for the light capturing area, which means that global shutter sensors are not so good in low light, all things being equal,  either. Furthermore, the added electronics complexity makes them more expensive to design and manufacture.

Global shutter sensors are much better for object identification and tracking, particularly if there's a fast moving subject, and it's also easier to read out high frame rates. So, you can see how such a design would be desirable to have with a modern auto-focus system and the demand for slow motion.

Nikon's patented sensor design

Now, a sensor design that can be switched between rolling shutter and global shutter modes is not a new thing and has been done before. In fact it features in some of the Phantom high speed cameras. However, an updated patent by Nikon for a sensor design that it has had in the works since at least 2016, and spotted by Asobinet, outlines a sensor design that can switch between rolling shutter and global shutter modes within a designated area of the sensor.

So for example, most of the image could still be in rolling shutter mode, but the area in which an object is being tracked could be placed in global shutter mode. It's certainly an interesting application, and although Nikon is unlikely to release such a sensor into the wild any time soon, it does pose some questions. One important question would be regarding the effect on the frame. If you were tracking a fast moving object, could there be some weird artefacts whereby some of the image exhibits skewing while the tracked object portion has no such effects? Such an image might appear a little 'weird'.

That said, it goes to show that innovation in sensor design is very much still alive and well, although I don't expect we'll be seeing such a design in a camera very soon.

H/T Petapixel.

tl;dr

• Nikon has proposed a sensor design that can switch between rolling shutter and global shutter modes within a designated area of the sensor.
• The global shutter design exposes and reads out the entire image in one go, eliminating the skewing effect seen in rolling shutter designs, but it requires more electronics on the chip, creating noise and heat.
• Global shutter sensors are better for object identification and tracking, making them desirable for modern auto-focus systems and slow motion capture.
• While the proposed sensor design presents an interesting application, it raises questions about potential artefacts when switching between shutter modes during tracking a fast moving object.