Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

@chayro You raise a good point. IMHE, the effect is magnified swapping components after just a track or two. I've seen this many times at local audio club and store demo events. 

When I do store demos, I find it much more helpful to listen for an hour or two, switch and replay the same tracks on the 2nd component. Sure, going weeks or more would be better, but excluding home demos, that's just not happening. Unfortunately, break-in issues make "30day return" home demos on new gear hard to judge too. 

Putting it all together usually leads me to buying used broken-in, depreciated gear when possible and considering my purchase an extended home demo at the cost of potentially reselling effort/cost. Cheers,

Spencer

Amir from Audio Science Review. Quite prolific and a great believer in measurements. 

I know. I usually mark my calendar for a month before making any conclusions. But it’s not just blind testing. By Amir’s logic, any back and forth testing would be invalid, except, I would imagine, in the most egregious of cases.  

I agree.  It can take months for me to fully appreciate an upgrade. Something I never noticed before in a passage or song I may not have listened to since getting the upgrade.  While the idea of blind testing sounds logical, I personally don't think it applies to upgrades or differences that can be deemed subtle.  Some differences can be discerned immediately, many I think not.