Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

Showing 4 responses by whipsaw

If what has been promoted was marketing and snake oil and the earth didn’t move for you? It is common, yes, if people are honest.

Alternatively, you may have preferences, and the reviewer or whoever it was that lead you to have this preconceived expectation, had other preferences.   And again, that is common.

Neither of those hypotheticals would explain why a cable might not meet expectations, as, according to many, they shouldn't make any difference at all!

 

Part of the problem with the blind-test brigade is that I have been in a position to be vulnerable to expectation/confirmation bias many times ((over ~40 years in the game), yet there have been more than a few that failed to produce the "expected" result.

I don’t doubt that such biases are frequently responsible for the positive reactions of audiophiles who are assessing newly acquired gear, and I am certainly not immune. But if it were so simple, and the biases so predictable and strong, then why have I, and countless others, no doubt, been disappointed with components when the opposite would be expected?

On a tangential note, regarding blind testing, I do not believe that listening to short passages is necessarily fair, or best-practice. I have been able to distinguish between components when allowed to listen to full songs with which I am very familiar, but not when confined to snippets. And I know that others have had similar experiences.

Earlier on this thread I mentioned a couple of the issues that I have with those who base their judgments almost exclusively on measurements, and what I consider to be some limitations of A/B testing as it is typically performed.

But as to Amir and ASR, I think that in many cases his work is both valuable and practical. As a current example, on Friday he tested a "Hi-Res" download recording from PS Audio. He tested both DSD and PCM. What he found, and clearly demonstrated, is that the recording is flawed, and that there were dubious decisions made during the process.

As one who is not well-versed in the intricacies of streaming, I learned some interesting things from his brief video. Beyond that, and this is typically the case, I learned even more by reading the many comments, as they included insights from very sharp, highly experienced people, some of whom are electrical engineers.

Take a look at the thread, and, if you’re in the same boat as me, you will be surprised at what you may learn (e.g. that DSD files are of dubious value!).

PS Audio DSD review

@pabs85 

Well, if the choice were binary, then yes, I'd agree that listening would trump measuring. This can be confirmed by the fact that not all components which measure very well necessarily sound good to all ears (e.g. a dry, analytical amp), and that some well-established, highly regarded designers (e.g. Nelson Pass) intentionally add distortion to their products, as their customers prefer the resulting sound signature(s).

Of course referring to both is arguably preferable, as measurements can, at a minimum, help consumers to avoid obviously flawed products.