In the second half of our two part series on digital equalizer design, we get into the nitty gritty of the errors that cause digital filtering to sound less than ideal. By Christian Luther.
Welcome to part two of a quick journey through the design of digital equalizers. If you haven't yet, check out part one for an introduction to the typical challenges of digital filter design which are related to the sampling process.
But time is not the only dimension in which audio is chopped up into small, computer-manageable pieces. The signal values themselves are also quantized and thus represented digitally with finite precision.
This numerical precision is, of course, also a limiting factor when computing digital filters, as opposed to just recording and playback. Each mathematical operation performed on the digital signal can introduce a roundoff error (not all operations do, though). In the end, it depends on the algorithm as to how serious this problem is.
The filter algorithms typically used in digital equalizers have two properties that can lead to problems with numerical precision. First, the coefficients (the numbers that define the behavior of the filter) can take numerically very adverse values. Simply put, this can lead to situations where very small intermediate values occur during the computation (or the difference of two numbers that are very close together). In this case, roundoff errors can get amplified by a large amount in the end. As a result, the numerical error can become much larger in level than that of the usual quantization noise you'd expect if you just record a signal at a certain bit depth (the often quoted '6dB per bit' figure).
Second, as the filter structures are recursive, which means that results from previous computations get fed back into the next computation, the error is also fed back and thus 'stays in the filter'.
Both effects can lead to serious problems, especially with the basic textbook filter structures called direct forms. The result can be increased noise, but also infinite ringing of the filter at low levels. By performing the computations at a higher bit precision and/or using filter algorithms that are more forgiving for roundoff errors, the problems can be reduced.
A funny thing about these roundoff error problems is that they're rarely very obvious. It often takes some experience and/or the help of measurements to notice them or some subsequent processing to make it noticeable at all. Moreover, although I called the result "noise," the errors are not the same thing as just an added noise component to the signal.
First of all, the noise only occurs when signal is present. It would be fair to say that the signal might mask the noise in this situation. But it also means the noise is modulated in level by the signal.
Second, the noise is correlated with the signal. That means if you would listen to the error in isolation, you would still recognize the original sound in a way.
Thus, the audible effect of these errors is not necessarily that you hear an annoying noisy artifact, it's also just 'different' than how it should ideally be. That's what makes it hard to put the finger on the exact problem when you hear it. It doesn't sound obviously 'broken', just different.
Finally, the fact that there are a lot of equalizer filters at work in a whole mix, errors can add up and the effect on the complete mix can be much more noticeable than on a single track. But at the same time, due to the complexity of a mixed piece of music, it can be even harder to clearly recognize what the problem could possibly be.
So what's the takeaway here? You see that even with simple standard equalizers, there are quite some pitfalls. Although there are several ways to improve the filter behavior, which are surely used in many of today's products, the basic textbook methods are probably still widespread.
Should you be worried now? Probably not too much. It is not like a non-optimal filter here and there will ruin your mix completely. As a general advice, you should trust your ears and your workflow and use the tools you are comfortable with. In the end, your intuition and experience will lead you to the right tools for the job, without figuring out how to test for the issues I described. But knowing what the differences are will help you to listen more closely and get a better sense of what makes a great equalizer.
Read more from Christian at his own site, http://science-of-sound.net
Equalizer graphic by Shutterstock