NB - all the following illustrations accompanying this article are full-size crops from HD frames with a full-frame insert to demonstrate the framing. Several of them are also an education on what compression does to grain, and although they demonstrate what needs to be demonstrated, I can only apologise to any directors of photography who may feel hard done by. The images will be most accurate if your browser is set to 100% scaling.
Modern technology allows us to get things right in camerawork much more easily than before. Resolution, sensitivity and noise performance have all improved enormously. It is easier than ever to shoot sharp, clean picture and, at almost the same time, an enthusiasm has exploded for what might be called – well – grunge.
To start with a familiar example, Saving Private Ryan actually isn't the grainiest film ever made, having been shot on Kodak's EXR 5293 stock. At EI 200, this may seem slow to people used to the speed of modern cameras, especially since it's a tungsten stock which would lose a bit less than a stop when used under daylight with an 85 filter. Effective sensitivity would be a bit more than half that; Kodak says 125, around two stops slower than a modern camera at the sort of ISO where many of them perform best.
The combination of this stock with the well-known bleach bypass process results in a film with grain that's visible, but not overbearing and was perhaps among the establishers of a trend towards the deliberately coarse. A contemporaneous film is Darren Aronofsky's first work Pi, which, as a practically surrealist piece, went for extremes on black-and-white reversal stock (Kodak's Plus-X and Tri-X, where even more grain and contrast was desired). Pi was made for practically nothing, and reversal is more expensive than negative, so it was very clearly a creative choice. Aronofsky and his director of photography Matthew Libatique, ASC could have had a cleaner image if they'd wanted it. They didn't.
Thus, after decades of research to minimise grain, after the late 1990s, grain was suddenly cool. From a purely technical point of view, grain is noise. It doesn't look the same as video noise and it's hard to compare the two, but ultimately it's detail in a recording that doesn't have anything to do with the characteristics of what it was trying to record, so it's noise. The edges of pixels are also noise – quantisation noise – as are the motion characteristic of 24-frame motion pictures.
Regardless, we are no longer required to strive for the absolute best results, as defined by clinical technologism. That 24-frame artifact is almost universally approved-of, so let's not get too sensitive about terminology. We can go back-and-forth about how much of this is a learned response to certain types of image and the stories they're associated with, but that's really for the psychologists. What matters is that it works – and what degree of what effects are appropriate at what times.
Issues of taste are, naturally, matters for the individual filmmaker and our subject is one of implementation. Many techniques use or imitate film; Black Swan (notably another Aronofsky/Libatique collaboration) shot a lot of 16mm because they wanted grain, but also because they wanted small, light cameras. Even so, it's clearly a deliberate choice: there are cameras smaller and lighter than super-16mm types that produce less grain, especially given harsh crop to 2.39:1 and the fast Fuji 500T that was used for some of the film. What's notable is that most of Black Swan takes place in present-day New York, used real locations and, in general, presents a realistic, almost documentary-style image, at least until Natalie Portman starts sprouting feathers. It doesn't have to be a surrealist art film.
So, you can shoot film. You could even shoot super-8, since modern scanning techniques have managed to extract results from it which come close to equalling the 16mm of old. Super-8 cameras can be a chore, however, and often it's about as expensive as 16mm. You can underexpose video and grade it back up, of course. This is an old technique, writ of the bad old days when Rec. 709 video was the only way and there was a need to keep highlights out of the hyper-compressed region at the top of most cameras' luminance handling curves. The problem is that video noise is stigmatised in a way that film noise isn't and can suffer terribly if compressed. If we're going to admit film grain, however, we should be willing to admit video noise and there's probably a perfectly reasonable artistic justification for it in some circumstances.
Finally, there's grain simulation. This has been done with varying levels of fidelity and success. Techniques from the simple application of noise all the way through to shooting magazine-loads of grey card on suitably grainy 35mm film stocks for later compositing have been tried. It's worth bearing in mind that this last technique, the superimposition of real film grain, is not a guaranteed route to accurate simulation. The visibility of film grain is greatly dependent on how well-exposed and how high contrast the image is. The technique is perfectly legitimate, it just requires proper compositing.
Perhaps the best techniques for from-scratch grain generation involve several layers of noise, blurred a pixel or two, then sharpened to create edges, with optional random displacement to create grains of different shapes. With each of a few layers scaled differently, a simulation can be achieved that isn't completely ridiculous. The saturation of the colour variation in the noise must be controlled and care must be taken to ensure that the grain fades off into shadows and highlights appropriately. Sometimes, this can be done by modulating the grain with the luminance of the target image or, sometimes, simply by crushing blacks on the resulting composite.
Done well, it can have much the same effect on an audience as the real thing. It can even go some way to making less attractive, real-world noise and compression artifacts look a lot prettier. Perhaps surprisingly, this sort of thing can be quite slow to render, so it can be worth making up brief, loopable clips of it. Perhaps the most powerful piece of advice with this sort of thing is to control the intensity of the grain by controlling its scale, not its transparency.
The only remaining challenge is in finding a way to prevent the video compression of almost all current distribution technologies from stomping all over your carefully-considered efforts, because sadly, most of them will.