Can we trust our own ears?

Woman with headphones graphic by www.shutterstock.com

Aside from reviewing noticeably compressed recordings, judging audio quality remains largely a subjective experience. By Roland Denning.

When it comes to making judgements about the quality of what we see and hear, few areas generate such heated arguments as the world of audio. Hi-fi enthusiasts will insist that what really matters is not technical measurements; what we must do is simply listen to the sound and judge for ourselves. That seems a perfectly reasonable approach – except that human beings are notoriously bad at subjective evaluation.

Hotly debated topics in the hi-fi world range from the engaging (high bitrates, vinyl vs. digital) to the contentious (bi-wiring, costly analogue cables, burning-in) to the outright ludicrous (hi-fi mains chords, plugs and fuses, green pens to mark the edges of CDs, directional cables). Why do we get so worked up about what should be one of life's great pleasures: listening to music?

The Placebo Affect

In the world of medicine, it is well accepted that placebos work. Put simply, if patients believe a pill will make them better, there is a good chance it will, even if the primary constituent of the pill is chalk. Additionally, blue placebo pills work better than white ones and red pills are the strongest of all. The more invasive the placebo, the more effect it has. Placebo surgery is, apparently, even more effective than pills.

A few years ago, tests showed that changing the colour of speaker grills would change the sound coming out of them – red grills will give the sound more bass, blue grills more clarity. Of course, there is no measurable difference in the sound produced, but, subjectively, the music changes.

What we see can change what we hear, even if we know what we are hearing remains the same. This demonstration of something known as the McGurk effect is a great example:

Play two pieces of music, identical except one is almost imperceptibly louder than the other (that is, not quite loud enough to be noticed as a volume difference) and listeners will invariably say that the slightly louder example sounds better (more detail, better bass etc.). It is an accepted phenomenon and one which have may have convinced a few people to buy an expensive DAC unit for their system.

Hi-fi aficionados will insist we should take time to evaluate equipment, we should listen to it in relaxed settings and we should take time to let new equipment to 'burn in'. Unfortunately, humans are notoriously bad at judging one sound against another and the longer the delay between switching between devices, the worse we are. If it takes 5 minutes to change from one hi-fi set up to another, our powers of accurate comparison are very weak indeed. 'Burning in' is largely a process of the listeners becoming accustomed to the idiosyncrasies of a system, not changes in the hardware.

If we have any pre-existing bias in favour of one piece of equipment over another (we are a fan of the brand, it looks more substantial, we have invested money in it already), it will sound better. We can not help that. This is called confirmation bias.

None of us are immune...

The important thing to realise is that none of us are immune from cognitive biases. This is nothing to be ashamed of, we are all human beings and human beings are just bad at subjective comparative evaluations. None of us like to be accused of making irrational judgements, but I am afraid we do it all the time. All of us.

The problem with audio is that, unlike, say, photography, we can't compare two samples at the same time. The only way to really compare audio is to use blind A/B testing, but some hardcore hi-fi devotees reject this, saying it creates an unnatural, clinical setting (an excuse those who claim extra sensory powers use when then they fail in laboratory tests). The anti-blind testing proponents get angered when an amplifier they 'know' to be superior fails to be recognised as such in blind testing – this, for them, disqualifies the scientific approach. Auditions in hi-fi showrooms, of course, are never blind.

For a long while, I was ripping CDs to my hard disc using lossless compression rather than high bit-rate mp3 or aac files. I was convinced the uncompressed files sounded better (scientifically, they should) – then I did my own blind test. There was a difference, but it was negligible – not enough to, in any way, lessen my enjoyment of the music. And more importantly, half the time I was comparing them, I thought the compressed files were the slightly better sounding ones. Now, you may say I have leaden ears or an inadequate hi-system or I'm simply too old – all quite reasonable comments – but I no longer bother using lossless compression (no dispute, however, that low bit-rate mp3 files sound inferior). I suggest you do your own blind tests, too. If you want to check how well you can tell the difference between 320kpbs and 128kpbs mp3 recordings, try this online blind test. The results might surprise you.

When Neil Young launched Pono, his 'ultra-high quality resolution' (192kHz/24-bit)  portable audio player last year, he claimed it would be a revelation, that for the first time fans would hear music as it sounded in the recording studio – a vast improvement on anything they would have heard before.

'High resolution' audio is, of course, a gift to marketing departments. Consumers have seen the undeniable difference between standard and high-definition video, so it is easy to sell an apparent audio equivalent. What's more, there are numbers to back it up (192 must be better than 44.1!). With a bit of encouragement, you might be able to get them to re-purchase all their music collection with high-resolution files.

The Pono is certainly a high-quality music player, but, in blind tests, the overwhelming majority of listeners can not distinguish between the CD quality (44.1kHz/16-bit) recordings and 'high resolution' recordings. Even those who argue in favour of the Pono have not claimed it sounds dramatically different; the quality of the original recording and the headphones used are far more important factors. That's not to say there is no difference, just that in real world situations, for the vast majority of people, there is nothing to be gained by replacing CDs with high resolution files.

But the placebo effect could be used to defend the hi-fi enthusiasts who reject objective measurement in favour of subjective evaluation; placebo medicines can 'work', even if they have no active ingredients. If the expensive, exotic cables appear to make your system sound better, even if objectively there is no measurable difference, couldn't we say they 'work' too? If you love collecting and playing vinyl, what does it matter if someone demonstrates that your system is theoretically inferior to digital? If you are really committed to the subjective approach, if you think it sounds better, then it does.

Illusion is our trade.

The fact that our ears and eyes can't be trusted should be embraced by filmmakers, for we deal in illusion. Film is dependent on the illusion of a series of still images creating the impression of movement; the irrational quirks of perception are what we work with all the time.

We know that high quality sound will make the pictures look better. We know that a lens with shallow depth of field will look sharper than one where everything is in focus, we know if we want to shoot a night scene, putting one bright light in make the rest of the scene appear darker. If we are grading and one scene has a lot of blue in and the following scene is neutral white or grey, the latter scene will tend to look yellow.  

If colour affects the way we perceive sound (and vice versa), this should come as no surprise to filmmakers. If changing the speaker grills to red to increase bass is viewed as self-delusion in a psychological experiment, in filmmaking combining mutually-enhancing colours and sounds are just techniques to make a scene stronger. If you are shooting a drama, you will use every trick in the book to make your villain more menacing and your protagonist more sympathetic. A music cue associated with a nefarious act can become sinister in itself, imbuing an otherwise neutral shot with a sense of dread.

So, let's use science to develop the best technology we can, then rely on our totally untrustworthy ears and eyes to enjoy it. I don't see much wrong in that.

Audio Graphic by Shutterstock