Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

Showing 21 responses by djones51

An audiophile switch, in my case the Etheregen, transformed my digital playback to the point where it became easy to listen to with almost every recording, in an analog way that I like to refer to as creamy.

Really? How do you know it's the switch? 

While I believe one could measure the jitter reduction of the unit,  if I were to measure the audio output of my DAC, I don’t believe the change would show up on an audio spectrum or waveform, or be measurable at all!

Think about this statement ? If it doesn't show up on the output of a DAC, in other words if the DAC shows the same measurement with or without this switch then what exactly is the switch doing that the DAC can't take care of? 

You can make all the subjective untested wonders of these various devices you like, I'll just file them with all the other subjective untested wonders produced by dubious devices.  

This has gone assbackwards the device under test is the listener not a DAC, amp, whatever. The measurements have shown no difference in two devices say DACs, person makes claim to hear differences, the ABX is for the person not the DAC. If the person can pick one DAC from the other better than chance, which is where statistics enter, then his claim is taken seriously and viewed as objective not subjective nonsense about blacker blacks and wider soundstages.

If a company wants to show their fuse, cable, smear on goop, DAC etc.... affects the sound then the type of blind tests being discussed using sufficient people shown to have excellent hearing etc.. would be conducted by a third party to avoid company influences. Not that any of these companies that claim this stuff will ever do them.

Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data?

No, unless it's not a proper conducted test.

Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.

Which is why in a properly controlled ABX test you can take as long as you like. 

I read somewhere Amir is a famous Egyptian movie star who grows prize winning watermelons in his spare time. It's a confirmed fact.

He is probably making good money (YouTube + Google) telling people with limited funds what they want to hear: money doesn’t make one happy.

Can show me the advertising on his YouTube. Thanks 

You just have to have an open mind and be ready to embrace the new.

I have an open mind. I’m perfectly willing to admit my ignorance on high priced cables, dacs, power "regenerators" , amps and how much improvements can attained with them. If any manufacturer of such item is willing to explain the science behind them, show the measurements and the results of independent testing I’ll happily change my mind. Until then I’ll keep an open mind but not so open my brains fall out.

I don't know where this notion of ears being a great tool, they're  "tools" cobbled together by millions of years of evolution they're simply good enough for us to survive. There are tools that hear way beyond human ears just as there are tools that see way beyond the human eye. 

and this includes the best tool of all, our ears. IOW, some use their ears better than others. Some, because of experience, training, open-mindedness, or simply physiological advantage hear better than others. Hard pill to swallow for some, but true.

Some use their ears and eyes not just their ears which means it's simply a subjective opinion that applies only to the one making the claim with the problem of bias not accounted for. It contributes nothing to gaining knowledge about a product. Hard pill to swallow for some, but true. 

What's the matter? Pill stuck in your throat? Correct, your subjective opinion using your ears and 'eyes' you conveniently forgot those ,  contributes nothing towards furthering knowledge about a product.  I am always amazed of the hilltop pontificating about use your ears yet nobody seems willing to do so. 

 

Everyone has their opinion and that's all it is including mine. The only way to get beyond opinions is with vigorous testing including our ears. You may hear better than I and 90% of the population but that tells us what? When you hear good enough to pick product A from B when they both measure beyond human audibility or measure within .1% variance using only your ears  in a controlled testing regimen then you've told us something useful.

This is true but it doesn't really tell us anything.

Some, because of experience, training, open-mindedness, or simply physiological advantage hear better than others.

The point of Amir's video mentioned in the OP is trying to get us past these notions of sighted listening being all that useful for anything other than personal preferences of the aesthetics of the item. Buy what you want, like what you want but personal preferences tells us nothing about the component only about 1 persons preferences. 

What is measured? Sinad? Look up how useful sinad is and you will see that it is not that useful.

SINAD is useful but it's not the only thing measured. It seems to me the amount of half truths and downright nonsense posted here about ASR borders on paranoia. Perhaps it's because they are a growing presence and some feel threatened but after all it's just another website the deals with measurements and objective and to some degree subject aspects of high fidelity playback systems. You dont have to take everything presented there as gospel but with an open mind you might surprise yourself as to what you might learn. 

Why do you start wetting your pants? There are some people, mostly young girls, who hear >20khz but I doubt anyone on this forum can. 

Amir's approach still assumes that he is measuring everything that matters in terms of ultimate sound quality

I don't see that, he's measuring the DUT and seeing if it does what the manufacturers claim it does. 

Amir talks up the value of unbiased listening tests, but then admits that he is clearly biased and unwilling to go to the trouble of setting up any unbiased listening tests himself.

Why would he set up BDX testing of a component that doesn't do what's claimed? It isn't his place to conduct a DBX but those who claim the component does things the measurements clearly show isn't possible. 

People are missing the point of these reviews. Amir tests the Product, shows the measurements are the same or different from like products or engineering standards . Explains hearing thresholds and says in HIS OPINION there likely is no or little difference in music reproduction and places the product on a scale of similarly measured products. It's not his job to tell you what to buy or if you can hear a difference but if you claim to hear a difference in similarly measured products the only way YOU can rule out YOUR inherent biases is with a properly controlled test. 

Speakers are measured on a Klippel and are usually listened to in mono.  Amir's listening part of the review is his opinion and in no way meant to influence your purchasing.  I don't own speakers recommended on ASR but I do appreciate the effort put forth by Amir in his speaker testing it gives one a good look as to if the speaker has been properly designed and how well it can be adjusted by DSP.  How you like speakers to sound is entirely your own preference. 

I'm not sure where this notion that audio precision is sceptical of their products ability to measure. Are they looking to improve their products? Certainly what company isn't. But if this idea has come about from a YouTube circulating from an engineer from AP then you need to watch it again. He isn't saying their device is lacking in measuring but that some companies cherry pick measurements performed by an AP device so consumers need to be wary when looking at company advertised SPECs which are not the measurements. 

Oh, the latest PS Audio YouTube is missing the objection to the device they aren't measuring the HC outlets. It's more smoke and mirrors from PS Audio  for their fanbois.

Mine flanks the TV as I don't really have anywhere else to put the speakers. Using REW  for measurements and listening I've never seen nor heard a difference with the TV or without. 

What a load of nonsense. Amir has no agenda. There are a wide variety of people who participate on the ASR forum from subjective tube amp users to nothing but the highest measurement users. I take that back there is an agenda exposing bull products like "audiophile switches, cables, fuses " and other highly questionable devices that serve no purpose . Get over yourselves and yeah, hopefully younger people getting into this hobby will have a bit of common sense and see BS $20K DACs and $10K cables for what they are jewelry.

Atkinson & Co. also have another advantage over ASR: he and his team actually listen to what they measure.

So does ASR and company.