Let’s try to focus for a second. There are many things that can go wrong with any audio test, even or perhaps especially blind tests. Failure to acknowledge that things can go wrong is an indication of the naive nature of blind test proponents. OK, so what can go wrong? Why do I say, “no test that has negative results means anything?”
Things that can go wrong
1. The system used for the test is not sufficiently resolving to distinguish differences that might be audible on a better system.
2. There are mistakes in the system that were not found even if there was a search for mistakes or errors. Saying that the test procedures are thorough doesn’t necessarily mean they really are thorough.
3. The hearing capability of the testee is not up to the task.
4. For a blind test involving many trials, say 10 or more, the odds are high that the testee doesn’t have the focus or strength to dinstinguish audible differences for periods of time without tiring. You can’t just say oh, well, that’s the way it goes.
5. System issues that go undetected due to naïveté of those involved in the test. Directionality of Fuses, directionality of cables, Polarity check of all connections, etc.
6. Using unfamiliar music for the test.
7. Using a test system unfamiliar to the testee.
8. Weather and other “external variables” that affect the sound of any system that could make hearing subtle differences in sound difficult or impossible.
What if results are positive?
If the results of a test are positive I would probably say the test was a success and the results were positive IN SPITE OF ALL THE THINGS THAT COULD HAVE GONE WRONG.
