Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

Showing 7 responses by emailists

I beg to differ that the “amateur with an analyzer” site is doing a service, because he makes errors that manufacturers point out.  
 

There are actual, ethical, knowledgeable people doing tech testing that serves the community.  Take this guy for instance, that many in the high end video monitor world respect, even the professional color grading forum I participate in mentions his reviews.  https://www.hdtvtest.co.uk

He has no agenda, unlike the amateur being discussed, who sells cheap gear.  
 

 

It appears that the fake science site are such complete amateurs that they measured the wrong AC output port on their recent power conditioner “review” according to Paul McGowen, who said they should issue a retraction for their shoddy work.

I call it the “idiot with an analyzer” phenomenon, but psychologists have a term for this kind of limited intelligence masquerading as knowledge.

“Dunning-Kruger effect, in psychology, a cognitive bias whereby people with limited knowledge or competence in a given intellectual or social domain greatly overestimate their own knowledge or competence in that domain relative to objective criteria or to the performance of their peers or of people in general.”

 

What people like @Prof are ignoring is that according to PS Audio, the fake science site (my term) measured the wrong AC output. Has anyone seen them publish a revision or retraction? Any legitimate site or journalistic source would immediately publish a retraction and retesting with new findings, Or at least a response from the manufacturer stating they measured the wrong thing. Will it happen? If a fake science site ignores their huge technical mistake, then they have no legitimately or integrity and completely invalidate their entire “testing and “review” process.
But hey, if it makes people feel good about the cheap gear they buy that “tests” better than more upmarket gear, it’s their prerogative to scream “snake oil” as I have seen on the few times I wandered over to that cesspool.

Also I have read in this thread that people don’t have a good aural memory, which is why AB testing is required. While this is most likely true, it’s not true for everyone. Many of the people in tests for cd quality vs. hires couldn’t hear a difference, but that doesn’t mean trained listeners can’t hear the difference, or can’t remember characteristics of sound over long periods.  

 

 

 

Regarding the PS "review." Science is based on replication and peer review.  If the  output used didn't measure correctly, why wouldn't a site with "science" in their name not test another outlet to verify results?  It's lazy.  Or reach out to the company and ask them about the different outlets.  I don't use Powerplants, so I can't comment on whether the manual is lacking information or not. Maybe the support forums clarify best use practices?  These are sophisticated products, and not necessarily plug and play.  

I reached out to the designer of my power conditioner to inquire about a few things, and I'm end end user, not a "reviewer."  

If someone wants to be seen as a legit source of unbiased information, they need to do the work to gain that level of respect, and a fake science site that (I believe) is also an audio dealer, is not that.

Here’s an actual smart person (who designed the powerplant) showing how to measure a power conditioner.

 

Here’s a perfect example. Teo Audio is truly innovating in the world of audio reproduction. But does your cable show measurable results of what it does?

It would be fantastic if it did, but my guess is that the human ear is needed to detect the benefit of your product, and it must be in an appropriate level system. Would you send them of your cables for “review?”. My guess is not. It would be like sending a Ferrari to a car review site whose criteria is gas mileage (but who can’t seem to consistently measure milage correctly, or worse proclaiming the Ferrari is inferior to the yugo, since it “tested” better. 

 

 

 

 

@djones51

wrote "exposing bull products like "audiophile switches, cables, fuses " and other highly questionable devices that serve no purpose"

An audiophile switch, in my case the Etheregen, transformed my digital playback to the point where it became easy to listen to with almost every recording, in an analog way that I like to refer to as creamy.

While I believe one could measure the jitter reduction of the unit, if I were to measure the audio output of my DAC, I don’t believe the change would show up on an audio spectrum or waveform, or be measurable at all!

I look at measurements as an afterthought and a guide to support what I might be hearing sonically, if I’m interested in a product.

I’ve posted previously Bascome King saying the best measuring amp he ever tested didn’t sound good.

The Etheregen was designed by John Swenson one of the top guys in the commercial switch industry, and took over two years to design and has advanced engineering and galvanic isolation.

I don’t get get - people who consider themselves audiophiles, but who seem to have their ears closed. If a product or tweak doesn’t benefit your system, that fine. It’s worthwhile reading people’s findings. But how can you claim that it isn’t benefitting my system and my listening experience?