2. misc-audio
  3. 175682-1635

Amir and Blind Testing

Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

Showing 10 responses by cleeds

cleeds
6,142 posts
 
 

cleeds Atkinson & Co. also have another advantage over ASR: he and his team actually listen to what they measure.

djones51 So does ASR and company.

Yes, sometimes he listens. Sometimes he doesn’t. At random, I pulled up this review. There’s no mention at all about how it sounds, but it earns a hearty recommendation just the same:

"The Gustard X16 comes in an unassuming package making you think you have bought a mid-priced DAC. But looks are deceiving as it produces some of the best measurements I have seen out of some 350+ DACs I have tested! Talk about getting ready to battle with the best of the best. Yes, there is a hint of ESS IMD Hump but it is not excessive enough for me to worry about it although I hope they look at eliminating this.

It is my pleasure to strongly recommend the Gustard X16 MQA DAC."

cleeds
6,142 posts
 

mahgister

If they listen they deceive themselves!

For sure.... 😁😊😊😊😊😊😊😋😎😊😊😊😊

The question is : they listen with a white suit and a measuring tool in their hand or they listen "nude and not afraid" ?

Ha, ha! But if only it were that simple with ASR. The fact is that he sometimes doesn't listen at all. So what's the point?

cleeds
6,142 posts
 
nonoise
...another link where someone pointed out the differences in the gear Amir uses for testing and what John Atkinson uses. It’s multiples of times more expensive and exacting, showing anomalies that Amir’s tester can’t.

Atkinson & Co. also have another advantage over ASR: he and his team actually listen to what they measure.

cleeds
6,142 posts
 
noske

 

Common sense suggests that something may be learnt from a test when there is a difference and 40% says there is none.

Perhaps. It may mean that 40 percent of the test subjects simply weren’t sufficiently interested in the test to really concentrate. (If you’ve ever been a subject in one of these tests, you likely found it tedious, at least after a while. Most subjects don’t find things like ABX listening tests to be fun.)

Perhaps the 40 percent of the subjects who heard no difference were distracted by the sirens of the passing fire trucks. Perhaps the appetizers set out to keep everyone comfy and nourished triggered gastric problems. Maybe the HVAC system wasn’t keeping up with the summer’s heat as the day wore on.

Please understand that I’m not dismissing the value of scientifically valid double-blind listening tests. I’ve been part of a few and they were interesting. But analyzing the data can be tricky business.

cleeds
6,142 posts
 
noske

 

If in fact there is a difference and 40% say there is none, this is saying that those 40% of people have less than optimal hearing.

The only thing your test would reveal is that under those test conditions, 40 percent of the subjects heard no difference. There’s no data to support your conclusion.

In fact, conducting a scientifically valid double-blind listening test is not as simple as some suspect. So if the test results are puzzling, you'd also have to consider whether the problem is with the test itself rather than the test subjects.

 

 

cleeds
6,142 posts
 
bubba_buoy
 
 
 
 
 
 

 

We cannot measure the difference between a middle C, played on a properly tuned clarinet vs. a synthesizer, vs. a trumpet.

It's trivially simple to measure the difference between a clarinet and a trumpet.

cleeds
6,142 posts
 
teo_audio
 

I could go on for pages and pages more, but the end point is that physical measurements as they stand, connected solely to engineering criteria and weighting, fail to take into consideration the capacities of the human ear and brain and generally fail to understand ’hard engineering based measurements’ limits and limited scope, in application, to the complex question at hand.

That is indeed the issue, @teo_audio, and it's one that divides our community. I'm among the undecided as to the extent measurements reveal what we hear, and that just infuriates the measurementalists who essentially insist that their "science" is already perfect. That separates them from the genuine scientists and engineers, who welcome inquiry.

That...is science. Not this illiterate tail wagging the dog we see on these forums, day in and day out.

Absolutely. And it doesn't even matter which "side" of the question you're on.

This is my beef with ASR. It is clearly anti-science. It is dogmatic.

It's absurd that anyone would pretend to evaluate audio equipment yet not listen to it as part of the process. But that reveals the extent of the measurementalist's faith and as with all matters of faith, no proof is required, by definition. But that sure isn't my religion.

cleeds
6,142 posts
 
mijostyn

 

The problem for many people is they trust their own perception.

It's a problem for you, apparently.

The result of all this is an entire industry based on deception.

That's amazing! An entire industry built on fraud! Yet - somehow - you've managed to pierce the deception and divine The Truth.

Yea, the real problem must be my perceptions. It can't possibly be anything else.

cleeds
6,142 posts
 
djones51
Everyone has their opinion and that's all it is including mine. The only way to get beyond opinions is with vigorous testing including our ears.

Yes, you've frequently issued these proclamations about your "One Way" of testing. But this is a hobbyist's group, not a scientific forum. No one here needs to satisfy your "rigorous testing" requirements.

... personal preferences tells us nothing about the component only about 1 persons preferences.

It's probably best that you speak just for yourself, rather than pretending to speak on behalf of some unnamed "us."

 

cleeds
6,142 posts
 
laoman

For someone to say you do not need to listen to a piece of equipment before buying it and also that the synergy of various components makes no difference is so funny.

Egg-zactly. But some people will buy a car without driving it first. Or they’ll buy a suit without having it properly tailored. It makes no sense.

More to discover