Why are digital streaming equipment manufacturers refusing to answer me?

Given the light discussion around ab(x) testing, as a scientist with phd training in human data collection and cognitive functioning, I object to the (x) part of the judgment task.

Identification should not be the goal. We listen to equipment to decide which we prefer... which is better.  That should be the decision when doing (blind) ab testing.

If I were designing the test for audio equipment, I'd let people switch back and forth as many times as they like, taking as long as they like for each (level matched) sample.  Eventually they just decide which they like best. Repeat that task over and over throughout the course of listening to music and I guarantee that people will begin to find their true preference and that that preference will be consistent. 

It's very easy for the brain to develop and make decisions based on good/bad/preference and much harder to make the decision based on label identification. They are different processes in the brain and preference is the more basic, a primary system. Newborns know what they like and what they don't and will often show good a/b consistency. The brain doesn't even need to identify a stimulus to make this judgment. That comes later because it's less important. First, figure out if something's good or bad, then if you've got time, figure out what it is.  This is the order of processing for all of us.

The eye doctor uses this ab-preference testing when they're figuring out your prescription. They flip between two possible magnification factors and you just tell them which is better. It works great for letting you find your own way to an optimal solution. They don't care about labels or giving you mystery options and having you say whether it corresponds to a or b because that's irrelevant. It would make the process less effective. 

And finally, I'll just say that if you don't believe in blind testing, you're lying to yourself. There's decades of research demonstrating the impossibility of avoiding the biasing effects of pre-existing knowledge. People have used samples of scientists trained in this area of research and they are just as influenced as you or I. You are not immune.

That said, it's hard as hell to do blind testing and I never do. Fortunately placebo effects are very real and they help your preconceived decisions feel right even if they were wrong.

Please tell me how matching is different than identification.

And please see above for a discussion of why the brain is better at determining preference than identification.

https://scholar.google.com/citations?user=NQGcNPQAAAAJ&hl=en

And just to highlight. With those memory patients, the preference test is highly reliable (defined in testing science as a consistent, repeatable judgment). Their performance on the identification test is completely unreliability (i.e., no different than chance). You do not need to be able to say x = a to make a reliable preference judgment.  I suspect if you instructed people to base their identification judgments in an abx solely on preference they'd do significantly better.

It's about saying a = x. That is an identification judgment. You don't have to believe me, but the brain is worse at that than at determining preference. 

They have studies with patients who have zero long term memory. Every day they fail an X test by failing to identify people with whom they've interacted with repeatedly.

Nonetheless, they form adaptive preferences for these individuals based on whether those past interactions have been positive or negative.  That is because the area of the brain responsible for preference is different (and more basic & important) than the area responsive for identification.

When people get the X test with audio equipment I suspect many are trying to figure out if the treble, bass, image depth, tonality, etc. match A or B, not asking themselves how much they like the presentation and then seeing if that preference is closer to how they felt when they listened to A or B. Of course we’ll never know if that’s true because unfortunately I don’t run an audio testing laboratory.

And yes, there is learning happening in the memory patients, but the area of the brain that makes explicit identification judgments does not have access to it. I’ll also note that when they fail to recognize a previous acquaintance that new/old judgment is even easier than matching to a particular object as required to succeed in ABx. Despite utter failure of explicit identification, the preference system chugs along just fine, leading to adaptive decision making.

This dissociation is why it is incorrect for you to state that, "if you can’t match A or B to X, then you can’t tell the two apart and you hence have no preference as you don’t as actually prefer either."

Anyway, it’s clear you’re going to continue to believe that matching judgments are the appropriate way to do A B testing. And that’s 100% not how I would do it if I were optimizing people’s decision making. It doesn't matter if you can pick your stereo equipment out of a lineup, it matters whether or not you like it. So be it...

@audio2design  Below is a journal article you might find interesting.  It explores a mechanism behind the empirical paradox that people can show a reliable preference between two stimuli but fail to discrimination between them on an ABX discrimination test (here referred to as triangle testing). 

I will note that the reason they identify in this case is actually "the statistical properties of the decision rules followed in different tasks."  I still suspect that raw preference judgments are more sensitive than discrimination judgments but that was not the driving factor for differences in this case.

https://link.springer.com/article/10.3758/BF03205304