Good point Newbee. There has to be a "reference". You must be listening to equipment that you're familiar with in an environment with which you're also familiar. I would tend to agree that "double-blind" testing performed by people who don't even know what they're listening for (Twl pointed this out elsewhere) is flawed at best and completely worthless.
What I'm referring to is conducting your own "in-home" blind testing where there is a point of reference and, hopefully, some relatively skilled ears listening for the most subtle of contrasts.
I've listened pretty extensively to my friend's system (almost as much as my own!) so I'm comfortable with using it as a reference. Additionally, my friend doesn't make nearly as many changes to his system as I do, so it's been pretty constant for a long time.
P_mmk also raises a good point about people make hyperbolic claims such as "the best" or "blows you away". I have heard people say such things about SACD which was one of the reasons that I wanted to conduct our informal test. While there was certainly a difference, it was not as obvious as I would have hoped.
Imagine the reaction people had to color television when it came out. Cable vs rabbit ears. Digital cable/satellite vs analog. DVD vs VHS. HDTV vs conventional broadcast. In the visual medium the changes are more obvious.
Ours is a hobby of incrementalism. This is all the more reason that we should be training our ears. It's easy to say that I hear a difference, but harder to prove. I'm not suggesting that I need to prove it to somebody else. I want to prove it to myself.
