Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

IMO if you have to do an A/B comparison blindfolded and struggle to hear any major differences, then keep what you Have.  If you don't hear an immediate improvement then cross that item off you list.

 

Happy Listening.

“Science is a bunch of guys in white coats cutting up frogs”. Woody Allen. But just to drag the thread back on topic, I really wasn’t talking about Amir’s measurements or reviews.  I was talking about his comments on the “analytical brain”  vs the “enjoying music” brain and the difficulty these may cause in evaluating components.  

"Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement."

It won't matter. If you're expecting a difference you will find that difference. Either in reality or imagination. And if you're really committed to the outcome, you will make an excuse for why you failed to find a difference or identify the correct device.

ABX testing allows you to listen and switch at your own pace with your own music, even over days or months, and still that has made no difference in the outcome of blind tests.

I participated in my first blind amp test in the late 80's and it was an eye-opener. Much along the lines of the infamous 1987 Stereo Review test.

@chayro , Those two issues, enjoying the music and analyzing what you are hearing are separate issues. I do not know about others but when I am analyzing what I am hearing I am not thinking about the music at all, my foot is not tapping. 

Unless I hear something I do not like when I am enjoying the music I am not in analysis mode. If I am trying to decide what I like better, say which copy of a specific album, I have usually gone about synchronizing the two and am switching back and forth listening to different aspects of the recordings. For personal reasons the most important factor is just being honest with yourself and expelling any bias that might enter the equation. 

It is hard to find someone who does not enjoy music but, fortunately for the world audiophiles are a much rarer breed. You can do both.

Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer. We are too affected by our biases to come up with any valid data. Maybe.

Actually, the human brain and ear ARE an spectrum analyzer. A FFT type spectrum analyzer. The world’s finest, in fact.

Here’s the kicker, though:

That depends upon the owner-operator conditions, though. Which vary.

Each one is built out of similar materiel. But, each has different capacities.

Akin to the difference in intelligence. High Ear-Q vs Low Ear-Q.

This is also tied to intelligence and capacities to understand the abstract in mental terms and descriptions. All the stuff required to make it to the point of doing correctly framed physical experiments. Experiments known to be limited in their scope and the weighing of such outcomes in the so-called real world..

As it all has individual aspects, in differential, and happens internally in the brain and ear and mind of the owner-operator of the given ear-brain. There is no winner-loser-etc in in any connected mundane analysis, like a person passing a finishing line first, or whatnot. That childlike simplicity in analysis (of this complexity of a question) is not possible, here. It will require a brain and balance to get to the meat of the matter.

Next item up for bids on "the intellectual analysis is finally correct" scientific comedy show (when it comes to the correct definition of the question), is, that we each train our individualized Extreme grade FFT analyzer independently of anyone else.

Next item that comes up, is that this individualized organic based FFT device is UNCONSCIOUSLY shifted about, wordlessly shifted about, it is not written on paper and then we flip a switch with loud proclamations that we are flipping a switch, or twirling a dial, and changing the measurement criteria scope or range - that anyone can check on. There is no place or spot where everyone can see a physical differential or change is simply not possible, and never really will be.

No first past the post no monopoly ownership of a game, no photo finish, no measurements taken with gear. Thus the lower tiers of mundane analysis are denied their grip on the idea of physical realities that all can access. This sort of thinking is denied presence, denied by the barrier of intellectualism and thought being required  --as part of the set of components involved.

Ground pounders are confounded into confusion and insistence. Rightfully so, IMO and IME.

So, we move to intellectual and internal analysis where philosophy and thought and verbal/written analysis and synthesis takes place. Our highest endeavors are connected to our world in these sort of areas. The physical, for some, the visible, for some, the hard facts, for some, are all seen as the final arbiters of reality for humanity ------ but this is not true and has NEVER been true.

We enter into a point where are forced to go back to the origins of scientific rigor and deal with the point that philosophy, extrapolation, and rigor in logic... are our finest tools in defining the rigors of science. Defining the directions and scope of the overall human endeavor.

The next step in explaining why hard measurements over that of what ears say..to explain why that view and position is wrong..that complexity... involves getting the people who think such ways to understand they are wrong -- this is the nut or core of the problem.

Such a scenario involves getting them to understand something that they are seemingly wired to not understand, something that they block. Which turns to philosophy and the rigors of psychology as tied to the wiring of the human mind.

THAT is the big nut we try to crack in these arguments that seem to go on forever.

Sometimes it seems more than a little bit hopeless.

If it is constantly explained but the details go ’whoosh!’ and the deniers go back to their position of ’The hard measured physical reality in front of us describes all’..we are not going to get very far, are we?

Every person in the world of audio comes up against this complex question and we each handle it differently. Some go on to understand it, some butt their head against it forever.

Some feel they’ve reached some completion in the hard physical measurements but the question remains in others, to a high degree. To a high enough degree that others commit to high levels of battle with the ’measurement crowd’, to prevent the measurement crowd from attempting to force a reality on all others.

~~~~~~~~~~~~~~~~~~~~~

I could go on for pages and pages more, but the end point is that physical measurements as they stand, connected solely to engineering criteria and weighting, fail to take into consideration the capacities of the human ear and brain and generally fail to understand ’hard engineering based measurements’ limits and limited scope, in application, to the complex question at hand.

I await the next ’whoosher’ to come in and attack me, when the opposite is more the real desire. I desire them to step back, and reform their hypothesis. to move to real scientific based investigation. NOT engineering based investigation.

As engineering based attempts of analysis comes AFTER scientific definition of a problem and it’s parameters in/for analysis......NOT before.

Go back, go back, go back. Restate your assumptions., check their veracity. If the resistance remains then go back, go back, go back....and find the missing points in your assumptions. Then reform a hypothesis, and then move to designing a testing regimen that encompasses all of the KNOWN or SUSPECTED question.

Where.. if one is lucky, the results of that may finally reflect the seen and known reality in all it’s complexities. The longer a question remains unanswered, the more fundamental the mistakes in the formation of it’s orignal hypothesis.

That...is science. Not this illiterate tail wagging the dog we see on these forums, day in and day out.

This is my beef with ASR. It is clearly anti-science. It is dogmatic. To a serious level of fault.