Why doesn't your video play smoothly?

Why on earth does your video not play smoothly on a computer?

It's subtle, but once you see it you can't unsee it. Subframe jitter causes unnatural motion, and you could be fooled into thinking it's your equipment at fault. But it isn't an easy problem to solve. 

“Oh, dear,” said someone, frowning at a monitor. “It's not very smooth, is it?” The thing that wasn't very smooth was a rolling title. The problem is visible if you look at any recently-produced rolling title on YouTube on a desktop computer. Avoid things transferred from film because film isn't entirely steady at the best of times and will obscure the effect, but the problem is often fairly visible. There's a sort of jitteriness to the image, right?

No matter how good the dolly, if the frames aren't played back at regular intervals, things won't be smooth

We don't usually notice it because we're used to it. We're not talking about video pausing because of inadequate network bandwidth or deliberately low frame rate, but a tiny jitteriness that only appears on things like phones and computers. It's most visible on things like rolling titles, but anything involving slow, even movement, such as a zoom or slow pan or tilt, can reveal it. But theoretically, smooth and even motion is relatively straightforward: if an object changes position by the same amount between a series of frames, it'll move at a smooth and even rate.

Well, as so often, not quite.

Techniques for moving something smoothly date back to the earliest days of electronically generated titles on analogue video. We'll have to digress a bit, so we can understand how complex it gets. Rolling titles were originally created by aiming a camera at a text on a card and moving either the camera or the card. Smoothness was limited only by the mechanism used to create the motion and the ability of the camera to avoid aliasing (which tube cameras did.)

The right character is one-half pixel higher than the left and appears very slightly thickened in its horizontal strokes. This can cause jitteriness in moving titles

Do it electronically and things seem easier. We can move the text exactly the same amount every frame and even adjust the speed for predictable timing. Although, actually, we can't. If we don't move the title a whole number of pixels per frame, the computer will have to use antialiasing to move it a fraction of a pixel. This can create a lumpen, pulsating look to things like text, as its alignment changes with respect to the grid of pixels on the screen. This is an even worse problem with interlaced video where the text needs to maintain its alignment with the lines of each field, so it can really only be moved by a multiple of two lines per frame.

Ultimately, there are only two or three speeds that really work well for rolling titles. Modern software is clever and very soft antialiasing can make anything look reasonable, but it costs a lot of resolution. This isn't great if you want to cram in every member of the CGI department on a superhero movie without visible artefacts crawling all over it, even at cinema resolutions.

Three strengths of antialiasing. Left is without antialiasing, centre and right use different techniques. Getting this right is one of the side issues of smooth titling since aliasing affects motion too

But that's not what we're really here to talk about. Now that we understand exactly how much effort is required to create something as apparently trivial as a smoothly-scrolling title, it's time to ponder why, in the knowledge of all this, things often look slightly jittery on computers.

It's because computers don't play things back at the right frame rate.

OK, on average, over many seconds, they do. They have to, or the audio would get out of sync. On a moment-to-moment basis, however, they're often all over the place. Let's consider a 24-frames video stream, during which each frame should be displayed for 41.33 milliseconds. Most computers will display that frame for either 33.66ms or 50ms, somewhat at random. They have no choice: most combinations of monitors and computers (and most tablets and cell phones) update their displays 60 times per second, even in 50Hz parts of the world. Display a 24p video frame for two monitor updates and it's displayed for 33.66ms. Display it for three monitor updates and visible for 50ms. 41.66 is not an option. This creates what we might call subframe jitter as seen on a lot of computer monitors.

Information as typically provided by a video playback application on a computer. The display refresh rate has nothing to do with the rate in the file and the player can't control it

It's tough to alter this behaviour on tablets and phones. Many computers will allow the user to alter the rate at which the display is updated to something like 72Hz, which will (usually) show each 24p frame exactly three times and look much smoother. People producing 25-frame productions for 50Hz locales have the option of 50Hz or 75Hz updates. Most Windows workstations can do this using supplied software, though Mac users may need the SwitchResX utility.

On many computers, it's possible to finely control the way the monitor works. Here we see Nvidia's control panel on a Windows workstation. It even supports interlacing

It won't be perfect. Video playback software usually relies on the audio hardware to control its playback rate. Duplicating audio samples creates a click, whereas extending video frames is hard to spot. The audio and display hardware use crystal timing, but they are not locked together. Over long periods they will run out of synchronisation with each other. As this happens, the occasional frame will get longer or shorter to maintain sound and picture sync.

The correct solution to this is something like a Blackmagic or AJA I/O board for the computer, which has both the audio and video hardware running from the same electronic synchronisation source and sends out a signal with one picture per frame, guaranteed. Making a Blu-ray and running it on a TV has the same effect since the Blu-ray player will create a similarly reliable signal with picture and sound locked together.

Ultimately, this is pretty much the second problem (after colour accuracy) with using desktop displays for video. It's doable, but we need to be aware of how to do it properly and that the very occasional long or short frame is inevitable. But then, it is when people play it on their phones, anyway.

Title image courtesy of Shutterstock.