2. misc-audio
  3. 175682-1635

Amir and Blind Testing

Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

Showing 2 responses by jssmith

jssmith
325 posts
 

@henry53 

In a recent test Amir 'beheaded' a product (i.e. lowest rating) because it had an unacceptable 0.003% distortion. Can anyone can hear 0.003% distortion, are any speakers possible of even 0.03% distortion? What about 0.3% distortion? I have since listened and bought the product, it sounds wonderful, several others agree, the measurements have spoken,  but what do they mean?

I've seen him downgrade a product because it didn't measure as well as the competition or it was substantially more expensive with no gain in value. A downgrade doesn't necessarily mean it doesn't sound good.

The opposite is also true. I've seen him recommend products that don't necessarily measure well (although the flaws are still inaudible), but are a great value.

 

jssmith
325 posts
 

"Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement."

It won't matter. If you're expecting a difference you will find that difference. Either in reality or imagination. And if you're really committed to the outcome, you will make an excuse for why you failed to find a difference or identify the correct device.

ABX testing allows you to listen and switch at your own pace with your own music, even over days or months, and still that has made no difference in the outcome of blind tests.

I participated in my first blind amp test in the late 80's and it was an eye-opener. Much along the lines of the infamous 1987 Stereo Review test.

More to discover