Why Do So Many Audiophiles Reject Blind Testing Of Audio Components?


Because it was scientifically proven to be useless more than 60 years ago.

A speech scientist by the name of Irwin Pollack have conducted an experiment in the early 1950s. In a blind ABX listening test, he asked people to distinguish minimal pairs of consonants (like “r” and “l”, or “t” and “p”).

He found out that listeners had no problem telling these consonants apart when they were played back immediately one after the other. But as he increased the pause between the playbacks, the listener’s ability to distinguish between them diminished. Once the time separating the sounds exceeded 10-15 milliseconds (approximately 1/100th of a second), people had a really hard time telling obviously different sounds apart. Their answers became statistically no better than a random guess.

If you are interested in the science of these things, here’s a nice summary:

Categorical and noncategorical modes of speech perception along the voicing continuum

Since then, the experiment was repeated many times (last major update in 2000, Reliability of a dichotic consonant-vowel pairs task using an ABX procedure.)

So reliably recognizing the difference between similar sounds in an ABX environment is impossible. 15ms playback gap, and the listener’s guess becomes no better than random. This happens because humans don't have any meaningful waveform memory. We cannot exactly recall the sound itself, and rely on various mental models for comparison. It takes time and effort to develop these models, thus making us really bad at playing "spot the sonic difference right now and here" game.

Also, please note that the experimenters were using the sounds of speech. Human ears have significantly better resolution and discrimination in the speech spectrum. If a comparison method is not working well with speech, it would not work at all with music.

So the “double blind testing” crowd is worshiping an ABX protocol that was scientifically proven more than 60 years ago to be completely unsuitable for telling similar sounds apart. And they insist all the other methods are “unscientific.”

The irony seems to be lost on them.

Why do so many audiophiles reject blind testing of audio components? - Quora
128x128artemus_5

Showing 8 responses by millercarbon

Marvelous!

A fine example of self contained subjectivism, and not to mention self confessed expectancy bias, as you could ever hope to find.

Outside those select reviews you agree with there’s not a single external reference point anywhere!

Are you seriously recommending this ’method’ in preference to blind listening tests??

Yes. And I am seriously recommending you work on your reading skills. For starters, when you see the words, "at this point" like were used above that is a clue to consider what came prior to "at this point". Next I would suggest you consider anyone can make an airtight counter-argument easy as pie provided only they are willing to disregard the actual argument in favor of one made up out of thin air. The people who study logic call this the fallacy of the straw man.

You built a great big straw man. Congratulations! Now watch what happens the first little breeze comes along:

https://systems.audiogon.com/systems/8367

"That was the best the Who album has ever sounded and I’ve heard it on a lot of different systems... I mean a lot!

That spot, the listening chair is a very special place that went beyond just listening to music. Great job creating a near religious experience."

"you’re a better man than me - sharing your work and discoveries and opening yourself up to ignorant comments. But thank you! I’ve learned much from you."

"Suffice it to say, it was the biggest, most powerful musical experience I’ve ever experienced in a home setting."

"WOW ! Is all I have to say. I don’t see anything out of place, in this system . Well thought out."

"The imaging was so good that I felt like the vocalist was performing right in front of me and that I could reach out and touch them."


Does that sound like "chasing my own tail"? Are all these people suffering from "expectancy bias" too?

It’s hard to pick a favorite but for some reason right now it would probably be this one:

"you’re a better man than me - sharing your work and discoveries and opening yourself up to ignorant comments. But thank you! I’ve learned much from you."

The world's first double-blind test was performed live on television before a national audience. https://youtu.be/gbNCBVzPYak?t=56
Over the course of the last 30 years my thinking on this has changed and evolved considerably. From the beginning in the 70's my view was you absolutely must go and listen. Then in the 90's this evolved to you must audition in your home. Because the results I got back then seemed entirely based on being able to do home audition.  

When it came time for a phono stage I took this to the max, doing home auditions on at least a dozen different phono stages. Whew! But it was all worth it in the end when I wound up with a ARC PH3SE that made me happy for a good 16 years.

All during those 16 years however my system continued to evolve. Everything got better. Including my review reading skills. When you get up into a very select range of components it simply is no longer practical to home audition.  

Also at this point I came to realize it makes little sense to go and listen. Why? Because components at a certain level, the very best ones aren't really doing anything. The perfect component does absolutely zero to make the sound good. Instead what happens is you realize a crap component can make everything crap, but a good component can do nothing to make it good again. All it can do is add zero additional crap.  

Therefore, if you find something really good and you go to listen it is only going to reveal all the other crap in the system. This is why I never bothered to go listen to Moabs even though I could have. If they are as good as I expect then I will only hear whatever they are connected to.

Now if you follow this line of reasoning to its logical conclusion you realize that going to hear something somewhere else is at best a two-edged sword. You may have a favorable impression, or not, but either way it can be just as easily based on everything else in the system and have little or nothing to do with what you think.

That is why for going on 15 years now I have not listened to or auditioned one single thing I have added to my system. Yet every single one of these additions has performed beyond expectations: Melody I880, Koetsu Black Goldline, Origin Live Conqueror, Teres Verus motor and controller, Herron VTPH2A phono stage, Townshend F1 cables, Pods, and Podiums, Tekton Moab, and a slew of others. Currently working on Raven Reflection MkIII.

So today my view has evolved to this: When you are new and starting out you absolutely must go and listen to as many different things as you can. Make the stores change components. Any component. They can switch power cords, interconnects, amps, speakers- does not matter. Just make them do it. Pay attention to how the sound changes when each component is changed. Compare this to as many reviews and user comments as possible.

When you gain experience try and do the same as much as you can in your own system. Bring a demo home, have friends bring stuff over, bring your stuff over to try in theirs. Try as many different things as you can.

Always with the goal of becoming so advanced and skilled you no longer have hardly any need for such things. Along the way you get good enough reading reviews, sifting through comments, and understanding what all the various components do and how they contribute to the overall sound, that you don't really even need to do this stuff any more.
The reason "so many audiophiles reject blind testing" is because blind testing is not for audiophiles. Blind testing is for designers, developers, and researchers.  

The only reason for an audiophile to be interested in blind testing is to prove something to some other audiophile. But there is nothing to prove! It would be like trying to "prove" that flour makes better gravy than corn starch. Do you need a double-blind test to "prove" that red wine is better after it has time to breathe? Why? If you disagree, simply swill it down. Right out of the bottle. Be my guest. 

Here is a little secret I will let you double-blind people in on: we all know you can't hear- and we don't care!
The one you mention, wretch2, not only should all his posts be removed but then he should be suspended. For a start. And this is from a guy who abhors censorship. But if the whole point of a forum is to exchange ideas to help people find what they need to build a better system, how can you do this with people clogging it up with nothing but blather, and worse? You can't. So they need to go.

The wretch is not being censored. There are no end of sites he can blather to his hearts content. Just not here.
That’s funny. Because I had no idea how cars work and so signed up for Auto Shop to learn. Every day the "class" was the teacher blathering some nonsense about maybe try and not skin your knuckles up too much today, I did that one time the wrench slipped boy did it hurt, yadayada blah blah freaking blah.

So I asked every kid there. They are all madly turning wrenches changing the carbs out and blathering banter. Not a one of them in the whole class the whole time had the foggiest idea how a car works! They were all monkeys with a hammer trying to fix a helicopter. Including the teacher!

So when I got a motorcycle and needed to tune it up I got a manual and learned myself. Two years later with a 240Z pulled the engine tore it down took it to a machine shop put it back together and it started on the first turn of the key, just fired up like it had never been taken apart. Even the timing was only very slightly off, because I had set it by eye very close which I was able to do because by this point I understand very well exactly what I am doing, how every single little bit of it works. All the stuff no one else was able to tell me about even in a class where that is supposed to be all they do.

If you want to learn this stuff the odds of running into someone like me who really does understand and can actually explain it are slim to none.

What that means is if you utter the words "double blind" in anything other than mockery and derision then YOU are the ape trying to fix the helicopter with a hammer. YOU need to drop the BS step away from the keyboard go out and DO and HEAR and LEARN- on your own.

I know it is scary having to actually do something. Yes it is a whole lot easier to jabber away on a keyboard pretending to know. But you know what? We can all clearly see that you do NOT know, because otherwise you would never utter the words double-blind in anything other than mockery and derision.
Short attention span theater intermission.

The subject was the premise double blind testing is inappropriate and meaningless. The supporting evidence is a study involving speech recognition. The flaw in the reasoning, no evidence is given that speech recognition and component evaluations involve similar brain functions. Let alone skills sets. So the OP premise is fatally flawed.

We now return you to our regularly scheduled blather.