In defense of ABX testing


We Audiophiles need to get ourselves out of the stoneage, reject mythology, and say goodbye to superstition. Especially the reviewers, who do us a disservice by endlessly writing articles claiming the latest tweak or gadget revolutionized the sound of their system. Likewise, any reviewer who claims that ABX testing is not applicable to high end audio needs to find a new career path. Like anything, there is a right way and many wrong ways. Hail Science!

Here's an interesting thread on the hydrogenaudio website:

http://www.hydrogenaud.io/forums/index.php?showtopic=108062

This caught my eye in particular:

"The problem with sighted evaluations is very visible in consumer high end audio, where all sorts of very poorly trained listeners claim that they have heard differences that, in technical terms are impossibly small or non existent.

The corresponding problem is that blind tests deal with this problem of false positives very effectively, but can easily produce false negatives."
psag
""The problem with sighted evaluations is very visible in consumer high end audio, where all sorts of very poorly trained listeners claim that they have heard differences that, in technical terms are impossibly small or non existent.

The corresponding problem is that blind tests deal with this problem of false positives very effectively, but can easily produce false negatives.""

I have the above quote as being from the thread you mentioned.

"Likewise, any reviewer who claims that ABX testing is not applicable to high end audio needs to find a new career path. Like anything, there is a right way and many wrong ways. Hail Science!"

That one appears to be yours. If not, tell me where it is because I didn't see it.
There is no such thing as a test that can be be generalized. Someone else may get entirely different results. Then which test is correct? And who decides?
ABX testing is stone-age personified, and Hydrogen Audio is where all the stone-agers gather. I guess I should add "IMO."
"01-15-15: Geoffkait
There is no such thing as a test that can be be generalized. Someone else may get entirely different results. Then which test is correct? And who decides?"

The whole issue is that there are no test's. There never have been and there probably never will be. The title of this thread is: "In defense of ABX testing". What testing? For years I've been asking these people to show me some of the tests they've done to back up what what they say. They can't. The best they can ever do is bring up concepts from psychology like expectation bias and just hang on to that like its the answer. lol. I have a degree in psychology. I know full well what these term mean and how they are applied. And if you think that you're dealing with a case of expectation bias, you still have to test for it. Otherwise you're just guessing.

The reason these guy's won't do any tests is because they know there's a really good chance they'll be wrong and they don't want to look bad. What's the first thing the OP says when I challenge him and his science?

"We Audiophiles need to get ourselves out of the stoneage, reject mythology, and say goodbye to superstition. Especially the reviewers, who do us a disservice by endlessly writing articles claiming the latest tweak or gadget revolutionized the sound of their system. Likewise, any reviewer who claims that ABX testing is not applicable to high end audio needs to find a new career path. Like anything, there is a right way and many wrong ways. Hail Science!"

"Actually they are not my words."

Well, he's probably right since you can't actually own words.
Zd542 wrote,

"For years I've been asking them to show me the tests."

I suspect maybe you've been asking the wrong people. Nobody is denying the difficulty in devising tests that work like they're supposed to, I.e. demonstrate whether A is better than B or whatever. Or whether some newfangled device is a fraud. Nobody is denying the existence of placebo effect or expectation bias or it's ugly sibling the reverse expectation bias or any other such psychological effects. But to declare that there are no proper tests is a little bit inaccurate.