Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

mahgister,

Thanks for alerting me to a typo.

I meant to write:  "I've been a member of the AVSforum forum for 20 years..."

(Not ASR).

 

 

When not handled properly, obsession with A/B tests is just as any another obsession turned into compulsion: harmful and self-sustaining. It ultimately stems from 1., lack of self confidence 2., lack of experience, 3., trust issues.

With these issues solved, it becomes meaningless whether you do a change blind or seeing. (Plus, you need follow up for it to work - break in period, different genres, susceptibility to line AC changes, etc, so just swapping as a form of A/B test is shallow at best.)

When listening to music / system, I just open myself to experience without pre-judging the system. Why? Because I heard cheap solutions punching heavy weight and ultra-bucket mortars turning out as duds, and one piece that sounded like a rust bucket singing in another system. I learned that we cannot bring a single preconception to a listening experience, and I have zero trouble accepting that.

So, go in with zero conceit and preconceptions. Seeing the box that I will hear will bias me to judge it AFTER the hearing on a different scale. (Should have done more or punched high above the weight).

After all you only truly need an AB test when you are a MANUFACTURER and you want an edge to sell.

 

What people like @Prof are ignoring is that according to PS Audio, the fake science site (my term) measured the wrong AC output. Has anyone seen them publish a revision or retraction? Any legitimate site or journalistic source would immediately publish a retraction and retesting with new findings, Or at least a response from the manufacturer stating they measured the wrong thing. Will it happen? If a fake science site ignores their huge technical mistake, then they have no legitimately or integrity and completely invalidate their entire “testing and “review” process.
But hey, if it makes people feel good about the cheap gear they buy that “tests” better than more upmarket gear, it’s their prerogative to scream “snake oil” as I have seen on the few times I wandered over to that cesspool.

Also I have read in this thread that people don’t have a good aural memory, which is why AB testing is required. While this is most likely true, it’s not true for everyone. Many of the people in tests for cd quality vs. hires couldn’t hear a difference, but that doesn’t mean trained listeners can’t hear the difference, or can’t remember characteristics of sound over long periods.  

 

 

 

How could anyone be a member at audio science review for over 20 years when they were established in 2016