Amir and Blind Testing

Dear Noske, Alexander Dumas once said something very wise: all generalisations are dangerous, including this one. If I am uncomfortable about something, it’s some guru telling people there is no spoon and capitalism is rotten. That is Amir. 

He believes that he has nailed what over fifty years of audio science have failed to grasp: that numbers tell the whole story. He believes, no he knows, that if he can’t measure a claim, it’s a lie. He knows there is no difference because his software and pro audio equipment (anybody uses that at home, by the way?) tells him so. Therefore there can be no difference. And although he concedes he can hear a difference, he tries to validate his thesis with a ridiculous mumbo jumbo of Matrix references, neuroscience and psychology.
 

If this is where you belong, enjoy the comfort of half truths. 

https://www.audiosciencereview.com/forum/index.php?threads/do-audiophile-network-switches-make-a-difference-video.20316/
 

I don’t understand Amir’s argument. I have huge experience listening, and I have a good idea about the perceptive ability to hear things, as I was an audio salesman for awhile.

 I hear differences in my system immediately after putting a new piece of equipment in. More or less details, differing tonal balance or emphasis in certain frequencies. Where the differences are small and thus difficult to ascertain (as when I auditioned the original Audience AU24 interconnect against the Audioquest King Cobra, each providing similar slightly warm tonal balance with similar detail), I don’t fret about it.

 I thought Amir went off the rails when he reviewed Schiit equipment, claiming that their products were dangerous. I don’t read him.

If you read my posts here in audiogon i NEVER rejected measures...

On the contrary i rejected objectivist and subjectivist opposition to be meaningless in psycho-acoustic where measures of different kind are CORRELATED always to subjective impressions...

Second: i promoted acoustic and psycho-acoustic EXPERIMENTS in each of my post against "subjective hyperbole" of reviewers when they spoke about their "taste" in costly gear changes.....I promoted embeddings electrical,mechanical and acoustical controls devices and experiments BEFORE any upgrade...Most upgrade are not so useful... Acoustic is...

I also suspect that our initial exposure to reproduced music informs our future choices.

Here you are right and Timbre recognition is the subjective METER by which we create our small room acoustic...

Unless one is Mahgister who has obviously figured everything out. Amir’s measurements are a resource and he does provide data with a methodology. Flawed, maybe, but better than subjective hyperbole.

Sorry but i dont think a bunch of measures so useful they can be for an ENGINEER would be useful for most people... But acoustic and psycho-acoustic knowledge will be in all case...

Measures are like food we eat them without always knowing what we eat and what for and MOST food content is transformed in waste at the end anyway...Acoustic experience is like air respiration , no waste........But we need to eat food anyway, you catch it ?

And keep your sarcasm about me for you....Thanks...

I figure out, yes, the acoustic of my OWN ROOM... This fact disturb you?

Try many hundred of listening experiments non stop on a 2 years period ( i am retired) and you will figure out yourself the basic and you will never need to give me your sarcasm anymore.....

@pabs85 

Well, if the choice were binary, then yes, I'd agree that listening would trump measuring. This can be confirmed by the fact that not all components which measure very well necessarily sound good to all ears (e.g. a dry, analytical amp), and that some well-established, highly regarded designers (e.g. Nelson Pass) intentionally add distortion to their products, as their customers prefer the resulting sound signature(s).

Of course referring to both is arguably preferable, as measurements can, at a minimum, help consumers to avoid obviously flawed products.

I also suspect that our initial exposure to reproduced music informs our future choices. Unless one is Mahgister who has obviously figured everything out. Amir's measurements are a resource and he does provide data with a methodology. Flawed, maybe, but better than subjective hyperbole. 

Great posts that say much in few words....

Amir and others on this thread are absolutely right: A/B comparisons are notoriously flawed by expectation bias; that’s just how our brains work. In my profession (drug discovery) we therefore use "double-blind" evaluations, where the experimenter (e.g. the audio dealer) and the patient (i.e. the customer) do not know whether they are receiving a new treatment, a standard treatment, or (in non-critical cases) a sugar pill (i.e. placebo). Only such an evaluation would either confirm or put in question Amir’s well-intended measurements, in the sense whether or not the data he measures are relevant to human musical enjoyment and thus would indicate - before you buy it - if a particular gear enhances or diminishes such pleasure (which is, I suppose, what this exercise should be all about). The respective measurements would indeed have to track with the "enjoyment score" after listening to a hidden piece of new gear or an old one, while the listener and the dealer would not even know what gear is being listened to. In that sense, Harry Pearson was correct in his criticism of both: lone reliance on measurements and on A/B comparisons. He also knew that a new piece of equipment might sound spectacular at the onset, only to become fatiguing after a few hours or even days, no matter how "good" the measured data were. Psychoacoustics were a budding discipline in his early days, and we are still just beginning to understand how we make esthetic decisions, and what important part THD plays in this puzzle, if any at all.

Psycho-acoustic is fundamental knowledge for tuning a room, physical acoustic is NOT ENOUGH, for example knowing the way our brain synthetize the information coming for our two ears from the two speakers and from the walls and the meaning of the delay in time between these two frontwaves and their timing reflections and the way we could use and timing them to differentiate each one of them for each ears... i used that to decide for an asymmetric distribution of my Helmholtz resonators from each speakers and around the room with succeess...

No need to A/B blind-testing here....Our ears biases or pleasure level internal meter in the recognition of the timbre and imaging experience and any other acoustical cue are our guide and are our ONLY teacher here...

Precisely what Amir want to erase i keep it : our ears biases history... He is like a children playing with tools without learning with his BODY....He does not know that science is multi-disciplinary...And measure in one field means not the same in another field... It is the reason why concepts are more important than mere numbers...

Obsession to confirm biases is not scientific inquiry... But using acquired biases could be a scientific tool ...

Amir appears believe we all suffer from expectation bias. One problem how does that explain when the expectation is for things to sound no better or worse but they don’t?

This part pertain to the "nocebo effect"... When you claim that there is no positive difference ever and could not be one... But like someone painting himself in a corner Amir did not know that at all.... 😁😊 He dont see the "beam" bias in his own eye...And he look for the bias straw in others eye...Comical...

Anyway i was insulted by many people the only time i go there not by Amir but by his disciples.... i was naive thinking people look for truth and improvement.... They look for confirmation biases yes.... But that include Amir’s disciples especially... My self i was looking about improving acoustic experience by correlating some minerals addition on some part of the gear.... Easy experiments to do....And easy to verify, by their impact on the timbre perception, less so easy to measure, i dont know...

Human hearing is able to detect very subtle change from the resonant body source of the sound, it is the way man create music and create his meaning... Some ignorant claim they could reduce this phenomena to decibel level measures for example to determine what is audible or not... Simplistic claim that say all about scientific complete ignorance of the neurophysiology of acoustic which is non linear to begin with... What is audible for human ears is qualitative features essential to our survival, not simple linearly measured numbers by simplistic tool but non linear interactive complex qualities resulting from the interaction between the sound source and the environment...

«Silence is a bias and a sound, especially when i speak with my wife»-Groucho Marx 🤓

You just have to have an open mind and be ready to embrace the new.

I have an open mind. I’m perfectly willing to admit my ignorance on high priced cables, dacs, power "regenerators" , amps and how much improvements can attained with them. If any manufacturer of such item is willing to explain the science behind them, show the measurements and the results of independent testing I’ll happily change my mind. Until then I’ll keep an open mind but not so open my brains fall out.

He is probably making good money (YouTube + Google) telling people with limited funds what they want to hear: money doesn’t make one happy.

Can show me the advertising on his YouTube. Thanks 

@reven6e Actually it was just a simple request, not an invitation to provide a summary on all the good things about ASR that audiophiles are uncomfortable with.

I'm also not familiar with the whole Matrix/spoon philosophy - do you have links at ASR for that I could have a look at?

So now I’m back thinking that going in paying attention to how you feel listening to the component rather than listening is likely a better way to test gear. 

And so we are back to square one.  This is exactly what audiophiles often do, and what blind testing is designed to eliminate.

Please supply a link to this video review.  I'd enjoy watching it, not least because I've not witnessed Amir bringing his wife in for a hearing experiment - was it an A/B blind test?

Sorry Noske, this was probably about a year ago. Not a regular on their forum and not a big fan. His Matrix borrowed “there is no spoon” mantra and all his psycho babble about expectation bias is in fact a business. He is probably making good money (YouTube + Google) telling people with limited funds what they want to hear: money doesn’t make one happy. The expensive conditioner is a con. Expensive cables are a con. Expensive DACs are a con. Expensive headphones are a con. And so it goes. You don’t actually have to read or watch any of his reviews because you know exactly what his conclusion is going to be. Singxer and Topping are awesome. Audiophile is a synonym for a fool.

If you enjoy this rhetoric do live your life in oblivion (to quote his favourite movie), learn nothing, keep your RME and Singxer with the conviction, no the certainty, that you have the best money can buy. And be happy. If however you ever want to try the red pill, I suggest you actually demo a power plant, a good DAC, a Melco S100. You don’t need an Amir to tell you how to hear and what to think. It’s easy enough to make up your own mind. You just have to have an open mind and be ready to embrace the new.

In my book spoons do exist. Otherwise we would eat like dogs. 

Reading through this I was agreeing with the original question, in that if we are testing gear then we are listening with our brain not heart, so of course everything would sound good. However, @whipsaw then made a great point about a component we went in expecting to like and it not delivering. I wanted and expected to like the Michi X3, I had a great deal available on it, and I couldn’t connect with it. So now I’m back thinking that going in paying attention to how you feel listening to the component rather than listening is likely a better way to test gear. 

Earlier on this thread I mentioned a couple of the issues that I have with those who base their judgments almost exclusively on measurements, and what I consider to be some limitations of A/B testing as it is typically performed.

But as to Amir and ASR, I think that in many cases his work is both valuable and practical. As a current example, on Friday he tested a "Hi-Res" download recording from PS Audio. He tested both DSD and PCM. What he found, and clearly demonstrated, is that the recording is flawed, and that there were dubious decisions made during the process.

As one who is not well-versed in the intricacies of streaming, I learned some interesting things from his brief video. Beyond that, and this is typically the case, I learned even more by reading the many comments, as they included insights from very sharp, highly experienced people, some of whom are electrical engineers.

Take a look at the thread, and, if you’re in the same boat as me, you will be surprised at what you may learn (e.g. that DSD files are of dubious value!).

PS Audio DSD review

Blind testing works for components that have a wider spread of differentiation. It works for "preferences" as in which spaghetti sauce do you like. You can tell that immediately.

The trouble w extended listening is audio memory is the shortest memory we have and differences are more nuanced. There, "preferential comparison" (flavor of presentation over weeks or months) might be harder to assess except to say "I never noticed that before" in terms of sound articulation.

Even this cannot be written in stone as everyone’s brain is different.

What we hear is often compelled by desire of what we want to hear (expectation bias).

There is an aspect that adds further confusion to the process...chance.  What would be an interesting experiment is if the aspect of "gamble" or "skin in the game" or "taking a chance" was eliminated because spending larger amounts of money on higher end stuff creates conflict of interest in my mind.

I cannot tell you the amount of times people coming into our salons, agonized over a decision when A/B (or C and D etc,) comparison differences were not as stand out between what they were listening to. Even taking things home for try out did not completely eliminate this conflict between money spent and objective/subjective auditory acquisition as the listener could not escape that big money was about to be spent at some point.

For the well healed, this aspect did not loom as large.

Amir appears believe we all suffer from expectation bias.  One  problem how does that explain when the expectation is for things to sound no better or worse but they don't?

@reven6e And guess what: he could clearly hear a difference. His wife could hear a difference. 

Please supply a link to this video review.  I'd enjoy watching it, not least because I've not witnessed Amir bringing his wife in for a hearing experiment - was it an A/B blind test?

I once wasted my time watching a video “review” of some audiophile switches. It was half an hour of blah blah blah about how his measurements demonstrated without a shadow of a doubt that there was absolutely no measurable difference between X Y and Z switches therefore it was impossible to hear any difference so it was all in your small “audiophile” brain, his mantra so to speak. Then he went on to briefly demo the switches he had just “measured” in his (joke of a) system. And guess what: he could clearly hear a difference. His wife could hear a difference. According to him the audiophile switches sounded significantly and audibly worse.
 

Without realising he had just contradicted and refuted his half hour case for “it’s in your head”. If he accepts that his experience contradicts his measurements, for better or worse, then his measurements are flawed. For me Amir became irrelevant after that video.

Our hearing, or actually our brain’s hearing center, adapts ... sort of like our eyes adapt to low light levels. And the hearing center has no memory, like the smell center has, when you smell something that you smelled once years ago, you know you smelled this before.

For an AB test to work it must be possible to make the switch instantly. At the very moment of the switch you can hear the difference ... if there is any.

After a while our brain takes over and we really don’t know what is happening in those grey cells. You can still decern if you like a particular sound, or if you don’t, over time, but to hear subtle differences, instantaneous switching is the only way.

But, wait - how do you evaluate the results of each test? How do the tests correlate to the SQ characteristics most important to you?

There are some tutorials available (both written and video) to help understand what the tests mean and therefore how they may be evaluated.

Understanding that might also answer your second question - there are things to watch out for, and are displayed on colorful pictures. I like pictures.

I also know that my preferences do not align well with many who contribute to ASR.

This is no cause to be disrespectful of the combined knowledge of the contributors, many of whom are electrical engineers, scientists and PhDs, and have contributed in some manner to design and building of audio and associated gears..

As with many things in life, it is valuable to learn the ability to pick and choose that which is useful to you.   

I believe that when Amir places an audio device on his test bench, he goes into "lab tech" mode, where sound only exists to confirm the perfection of his measurements.

Seriously, improvement is a hard thing to quantify.  If everybody agrees A > B, life is easy.  But what if you're the only one who thinks so?

In theory, this is a great advantage for the objectivist.  Instead of days of pain-staking listening, they spend a few hours running a battery of tests.  But, wait - how do you evaluate the results of each test?  How do the tests correlate to the SQ characteristics most important to you?

Some people need that kind of assurance, so ASR definitely serves a purpose.  However, if you're trying to decide if you'll like the SQ of a specific component in your particular setup playing your kind of music, I doubt that ASR will be much help.

The psychology of hearing is a huge subject, upon which many books have been written.

It may help audiophiles to deploy passive listening techniques when auditioning new gear. With the new component in my system, I play a recording at the normal volume for my taste, then go into the next door room and read a book about a completely different topic to audio or music. If my attention is diverted from the reading, to the music, by something that my brain has not identified before, then the new component may be contributing something interesting.

I deployed this technique when I purchased my last DAC five years ago. I auditioned 6 DAC's and only one lifted my head from the book I was reading. I still have that DAC in my system today.

Happy auditioning!

Amir is the high priest 

He is also a French-Israeli singer and songwriter.  A man of many talents.

Amir is the high priest of the ASR cult.

Years ago, they were militantly claiming that changes to the power supply of dvd/blu-ray players could not possibly have any effect on the video output, because bits are bits and the power supply does not effect the bits.

It was a theoretical argument, which was easily disproven by doing any changes from SMPS to LPS, which on the meter would measure visible changes in brightness levels, let alone color measurement differences and detail improvements.

Amir and co were also proclaiming Bybee devices would have no effect whatsoever on audio. Then one of them actually measured them, and the Bybee devices were reducing distortion. So did they admit they were wrong? Of course not, Amir just immediately moved the goalposts to saying the measured reduction could not possibly be audible, and then demanded double blind tests as proof the Bybee did anything. Then they got embarrassed and quickly locked the thread on their forums so it would move off the 1st page.

That’s who Amir and ASR are, they are measurement cultists, but with an utter arrogance regarding things they know nothing about but pretend to understand.

If I want my TV ISF calibrated, sure Amir or someone like him is the first person I would ask, but for everything else in this hobby be very wary about their proclaimed expertise.

He also knew that a new piece of equipment might sound spectacular at the onset, only to become fatiguing after a few hours or even days, no matter how "good" the measured data were.

This fatigue aspect is an issue with audio.. Not just with audio, but I digress.

This is where describing measurements as good, bad, or anything else is incorrect. It is data.

What can be read into the data matters.

The characteristics of amplification which contribute to fatigue may be measured and therefore predicted.

You mention THD amongst other things - yes, and some aspects are pleasing, and others are grating to the brain. (And some serve to mask certain issues in the recording process, but that’s another topic).

This is perhaps one reason why measurements are preferred over blind testing..

As for blind testing, however messy it is even at the best of times, I would lower the threshold to exclude the enjoyment or pleasing factor. Does it sound different? is a more realistic objective.

Measurements provided by Amir indicate to me which bits of gears I may or may not enjoy owning. Others have different preferences. The data in itself is neither good or bad - it is information.

I remember reading somewhere (and now I forget where) that A/B testing relies on our short term memory, which isn't the best method.

Relying on short term memory could prove problematic for certain individuals in some demographics.

Would you mind repeating your question?  I had to think about it.

Amir and others on this thread are absolutely right: A/B comparisons are notoriously flawed by expectation bias; that's just how our brains work. In my profession (drug discovery) we therefore use "double-blind" evaluations, where the experimenter (e.g. the audio dealer) and the patient (i.e. the customer) do not know whether they are receiving a new treatment, a standard treatment, or (in non-critical cases) a sugar pill (i.e. placebo). Only such an evaluation would either confirm or put in question Amir's well-intended measurements, in the sense whether or not the data he measures are relevant to human musical enjoyment and thus would indicate - before you buy it - if a particular gear enhances or diminishes such pleasure (which is, I suppose, what this exercise should be all about). The respective measurements would indeed have to track with the "enjoyment score" after listening to a hidden piece of new gear or an old one, while the listener and the dealer would not even know what gear is being listened to. In that sense, Harry Pearson was correct in his criticism of both: lone reliance on measurements and on A/B comparisons. He also knew that a new piece of equipment might sound spectacular at the onset, only to become fatiguing after a few hours or even days, no matter how "good" the measured data were. Psychoacoustics were a budding discipline in his early days, and we are still just beginning to understand how we make esthetic decisions, and what important part THD plays in this puzzle, if any at all.

I remember reading somewhere (and now I forget where) that A/B testing relies on our short term memory, which isn't the best method.

Not sure if that's true or not, but I felt it was an interesting point. But that would at least explain why a lot of people like to take their time before making decisions on whether something sounds better/different/worse.

I read somewhere Amir is a famous Egyptian movie star who grows prize winning watermelons in his spare time. It's a confirmed fact.

This anecdote has merit as a credible and verifiable snippet of gossip (for the erudite, technically known as hearsay).

Unless one does an actual blind test, every, and I mean every change will only result in a subjective change and won't stand up to scrutiny.

I read somewhere Amir is a famous Egyptian movie star who grows prize winning watermelons in his spare time. It's a confirmed fact.

which is confirmed by the fact that anything sold by the parent company Aoshida will get favourable reviews from Amir..

I have read this somewhere too.

which is confirmed by the fact that anything sold by the parent company Aoshida will get favourable reviews from Amir..

Don't forget also that Confirmation Bias works in both directions.

Your brain can "confirm" improvements it expects to hear.

Your brain can also "confirm" that you don't expect to hear any audible improvements.

All of which leads me to believe that most "upgrades" are subtle.

You need to make bigger jumps with your upgrades. 

It is nice to know your gear measures reasonably well.  For example, over the years I have come to realize I seem to like the sound of amplifiers that double (or almost) power from 8 ohms to 4 ohms.  Also, having a DAC with a high’ish S/N ratio is typically a good thing.  However in the context of a home audio system, what really matters is how things sound to you when you have it all put together.  I agree with those who leave new stuff in their system for a relatively long time and then take the new stuff out, put things back the way they were, and then listen some more, rather than doing quicker A/B listening comparisons.  I also believe level matching is important and actually critical when doing shorter A/B comparisons.  In cases of subtle differences resulting from tweaks, doodads, and cables, I mostly try not to sweat it and ultimately decide whether the item is even worth the money.  The cognitive bias involved in all of this is directly proportional to the amount of audio forums and equipment reviews the listener reads.

The analytical mind isn't binary.  It's not on vs. off, but more a matter of degrees.  Sometimes the mind, regardless of listening mode, has a clearly formed opinion and other times things are kind of opaque.  All of which leads me to believe that most "upgrades" are subtle.

If it takes you months to make an evaluation, then I think it's just as much a reflection of your mind and body's movement over time than the actual change in your system.  Your weight, blood pressure, eating habits, sleep patterns, sexual activity, exercise, drug usage (both medicinal and recreational), illness, etc. all have a direct bearing upon what you think and how you feel about the sound of music.

Does the system change make a difference?  Do you like the difference?  The words of the poet illuminate:

First there is a mountain, then there is no mountain, then there is.

You cannot tune a room by blindfolding each step of your process...

You cannot fine tune sets of springs under and on top of each speakers damped by concrete load in my case, you cannot fine tune this at near 100 gram of precise loading weight blinfolded...

You cannot put in place 40 pieces of shungite and copper all along your electrical grid blind folded it is unnecessary to begins with...

And all others of my various devices like Schumann resonators grid i dontneed to be blindfolded to test it.... It is enough to set them off or on, each one at a times..And listen to the huge final difference...

Same process for my grid of different ionizers ....Samething for my other devices...

Blindtest is a tool for industrial reasearch yes and for marketing purpose also yes, it could be interesting to play with it in audiophile experience why not? it is fun yes...

But it is USELESS for my embeddings controls journey in mechanical,electrical and acoustical experiments in my own room...It will make my journey impossible to even start because of the complex burden which is related to scientific blind testing protocol.... 😁😊

It is ignorance to promote systematic blind test in audiophile tuning....Any audiophile can put off or put on something and listen and decide erroneously in some case but anyway being right in most cases... . It will be impossible anyway to blind test EVERYTHING...Perhaps some of my devices is not anything than a placebo effect for sure, but that did not explain the HUGE S.Q. improvement i enjoy now after many hundred of SMALL changes...

Our goal is creating a pleasurable acoustic experience of music in a system/room synergy for our own ears..

Our goal is not to set the OBJECTIVE standard for an industrial product where blind testing is a crucial step...

Objectivist and subjectivist division is a childish division who make no sense whatsoever in acoustic and in psycho-acoustic where only the correlation of each OBJECTIVE change or measure is CORRELATED under control by a guiding SUBJECTIVE judgment...

The important word here is not subjective nor objective but controlled correlation...

And blind test is a form of controlled correlation yes... But reducing all form of controlled correlation to thisone only is foolish... Especially in the incremental process of acoustic tuning of our room which may implicate many hundred of small changes by listening experiments...

For The measures problem: all is measurable in principle but we cannot measures all that can be measured or all that must be measured  anyway and in some case we dont know what to measure... And anyway no one has the same ears/brain history and stuctural working... We differ... Then to set a room and tuning it  measures are necessary but are only tags all along the road not the road itself...

I have to chime in here just to say that I am gratified to see an actual discussion of a topic as opposed to the “wresting-match” mentality these things often take on. DCS vs MSB. Bruno Sammartino vs Dr. Tanaka, etc.  Too much of a battle scenario for my taste. Obviously, I was a fan in my younger days, but I prefer a more civilized tone for audio. Just my opinion.

Amir limited argument apply only for ONE change...

In an incremental ongoing process of MULTIPLE ORIENTED changes like adding multiple devices to controls the mechanical, electrical and acoustical working dimensions of our audio system, and for the tuning process of our room/speakers, our own LEARNED biases and audio history are an advantage and a tool and they are also our perspective peripheral original vision who make us able to focus on some center of attention at will... It is not only an analytical habit it is ALSO a synthesis of our sound interpretative history : a learned set of biases...

All biases are not always bad and a spectral analyser always right...it is not so simplistic...Science is not playing with tools....It is conceptual and experimental...

Acoustic is the main matter in audio not electronic engineering...Sorry for those who dont know this elementary fact ....

Let Amir play with his tools and we will set or system/room like we will like to enjoy it for music...I buy nothing, never upgrade and create my own devices... And i am satisfied ....I dont need obsessed tools fanatics to teach me that adding a quartz crystal+shungite and copper on a cable is stupid... I know what my ears teach me....All my devices cost peanuts but my system dont sound like peanuts...

Why there is not GENERAL acoustic thread in audio forums save precise questions sometimes by some but no GENERAL acoustic thread... I started one and it goes dead in days....

I will say why?

It is because people like Amir and the so called subjectivist think that it is electronic engineering that gives to us sound experience... It is not even wrong.... For me it is IGNORANCE or a lie... It is acoustic and psycho-acoustic that explain sound and give us music and sound higher experience... Not upgrading with the advice of Amir or from a fan of a brand name product...

My best to all....

Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer. We are too affected by our biases to come up with any valid data. Maybe.

Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement. 

I can get the feel of a new piece of gear in a few days but it takes longer than that to play enough different pieces of music to get the final answer. After a few days, it would be hard to fool your brain into thinking that something was better if it is not. Long term testing is the only way equipment can be reviewed. 

Neither of those hypotheticals would explain why a cable might not meet expectations, as, according to many, they shouldn’t make any difference at all!

My two possibilities are clearly not the only reasons, I just threw in a couple suggestions.

What do you suggest could be a reason that a cable may not meet expectations in the context of this thread?  Is this a common experience?

Incidentally, the level matching thing I wrote about earlier may not be such an issue with cables, although someone with specific experience may correct me.

If what has been promoted was marketing and snake oil and the earth didn’t move for you? It is common, yes, if people are honest.

Alternatively, you may have preferences, and the reviewer or whoever it was that lead you to have this preconceived expectation, had other preferences.   And again, that is common.

Neither of those hypotheticals would explain why a cable might not meet expectations, as, according to many, they shouldn't make any difference at all!

But if it were so simple, and the bias so strong, then why have I, and countless others, no doubt, been disappointed with components when the opposite would be expected?

If what has been promoted was marketing and snake oil and the earth didn’t move for you? It is common, yes, if people are honest.

Alternatively, you may have preferences, and the reviewer/influencer or whoever it was that lead you to have this preconceived expectation, had other preferences. And again, that is common.

I’d like to think these observations are reasonably self-evident, but perhaps not.

Part of the problem with the blind-test brigade is that I have been in a position to be vulnerable to expectation/confirmation bias many times ((over ~40 years in the game), yet there have been more than a few that failed to produce the "expected" result.

I don’t doubt that such biases are frequently responsible for the positive reactions of audiophiles who are assessing newly acquired gear, and I am certainly not immune. But if it were so simple, and the biases so predictable and strong, then why have I, and countless others, no doubt, been disappointed with components when the opposite would be expected?

On a tangential note, regarding blind testing, I do not believe that listening to short passages is necessarily fair, or best-practice. I have been able to distinguish between components when allowed to listen to full songs with which I am very familiar, but not when confined to snippets. And I know that others have had similar experiences.

Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data?

The data is the signal. The brain is what does the analysing of this signal, not the feeding.

Some form of blindfolding is necessary by definition, and a near instantaneous switching mechanism needs to be used. In a perfect test environment you wouldn’t be aware of when it has been switched back and forth.

Together with dB level matching - that might sometimes be the tricky bit (that’s an understatement), and the nuances that have been noted may well be because this step isn’t followed correctly.

That’s a good start.

Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data?

No, unless it's not a proper conducted test.

Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.

Which is why in a properly controlled ABX test you can take as long as you like. 

@chayro 

Great topic. You have accurately described the problems which arise when trying to assess a new component. IMO the next step is to find ways to overcome the biases and desires of our mindset. Lets face it, when we spend $1, 2, 5, 8K on an upgrade we want a return on our investment. And if one isn't there, we can easily fool ourselves into believing it IS there. I've done it. And I suspect many others have done so as well.

But you and others have given the answer to the problem. @spenav  has described my (and others) method saying,

It takes me months to assess a new piece of hardware and I usually won’t know until I remove it and see if I missed it.

This seems to be the method of you and most every poster so far. I  don't take months to assess an item as others have said, but weeks for me. And like spenav,

I usually won’t know until I remove it and see if I missed it.

That, for me, is the key. I believe it is easier to see or hear what we've missed than to hear what was gained, though that is in a mental database for comparison

Measurements are useful to me just to understand how and why my electronics are doing what they do. But our ears are incredibly sensitive to nuances that can't be measured. I think many audiophiles have experienced the joy of adding a component that is a huge upgrade to their ears in sound quality but measurements say it shouldn't be so.

  A recent thread about Carver amps comes to mind.They don't measure as expected and many owners were unpleasantly surprised, but the sound quality to them is outstanding. Having the information is a good thing but not the most important thing.

 If I find myself stuck in analytical mode it's time for a couple of days off,then go back later and just listen and relax to a wide variety of music.

@spenav- thank you for the compliment, but it was Amir who actually articulated it when he spoke about the analytical brain as opposed to the music-listening brain. Like you, I have felt this for a long time, but couldn’t crystallize the idea as well as Amir did. And while we’re here, I think we should stop using the word “improvement” and substitute “the component that sounds better to me”, which is, I think, what we really mean. 

@chayro 

You just say aloud something that a lot of us have known for a long time but couldn’t articulate as well as you did. It takes me months to assess a new piece of hardware and I usually won’t know until I remove it and see if I missed it. 
There is nothing wrong with measurements. That’s where it starts but listening has to be the ultimate value judgement. We always have to make sure we are measuring the right thing and only listening can validate that.