cleeds, I do not think it would be that difficult at all to design a randomized, double blinded study for audiophiles. I'm not saying it wouldn't be a little complex but it wouldn't be hard.
The issue is who would need such a test and why. I do not think it is useful for an individual audiophile for whom the only determinant is "what do I like better".
However, for critics (whose descriptions of equipment SQ all seem to sound the same) or for magazines/web sites that compare equipment I think such tests would be very valuable and legitimizing if results corroborate critical assessments. I deeply suspect that they wouldn't.....and that's why it isn't done.
Many areas in medicine that were once dominated by the diagnostic proficiency of experience physicians have gone by the wayside because studies show that many aspects of the physical exam are not nearly as sensitive or specific as physicians claimed and swore that they were.
4hannons, if I think two pieces of equipment sound the same then I'm done. Why spend time trying to hear a difference that is so minute as to be irrelevant? As you say, the key word is "relevant" I suppose.