Clever Little Clock - high-end audio insanity?

I'm copying below excerpts from a positive feedback review in a blind test *without* the CLC, from www.positive-feedback.com/Issue23/clock_nespa.htm:

Well, in a word, everything sounded bad. I was surprised at how awful things sounded.... ....Sadly, this song sounded the worst of all. Tinny and bright, and did not produce the chills down my spine that it usually does. ... I literally cringed ...sounded better when I was outside the listening room .... I turned it off completely, and that is my favorite track on the disc.

ok, so I'm supposed to believe that in their audiophile system the music was completely unbearable without the CLC, but when the CLC was brought into the room it sounded glorius. This defies logic and common sense - presumably she had been enjoying music in their home system before this blind test, but suddenly it became unbearable just before the CLC was brought into the house for a test? The way they staged their test also lacks credibility - why not do a true blind test? Sounds like complete nonsense.

So... the implications of this are: I can't trust audiophile magazines and I can't trust what other people report hearing. I can only trust my own ears. The problem is that I have very limited opportunities to audition components in my home, and it requires some expense and effort to arrange auditions. If I can't trust reviews then I don't have much basis for deciding when to go to the effort to arrange an audition. IMO this taints the entire highend audio community, I'll always have to wonder who is full of BS and who might actually be doing something worthwhile.

Has anyone done a true blind test - where someone else removes or inserts the CLC in your house? If you can reliably determine whether the CLC is anywhere in the house this would provide remarkable evidence for some totally unknown natural phenomenon. I have to confess that I find the concept so improbable that insanity seems like the more likely explanation - but if people can reliably detect its presence in blind tests I'd happily eat my words and order one.

Given the dramatic nature of the claims for the CLC (per the Positive Feedback review) it should be fairly easy to show that it works in a true blind test. It's also an incredibly easy test to do - no need to mess with interconnects or move speakers or amps around.

According to the Positive Feedback review the effects were so dramatic that it should be easy to carry out a succesful double blind test. Given how improbable this device is, I think the Clarks and Positive Feedback have an obligation to carry out this test to save their reputation. I'm trying to keep an open mind, but there seems to be a excuse to explain away every question.

I'm disappointed about what this says about high end audio in general - manufactures and reviewers are not to be trusted. Seems to me it hurts the industry and it discourages people from getting involved in the hobby.

I had a guest staying with me for a few days, so I put him the sweet spot and played some tunes, I loved his reaction, he just kept repeating, in a tone of amazement: "You don't need drugs, you don't need drugs..."

I teach research methods and find the question of whether a random sample's sampling error could have accounted for the variation noted to be trivial, especially as anything will be statistically significant if the sample size is large.

Maybe I misunderstood what you were trying to say, but to put it simply, if your sample size is too small, say 2 experiments with the CLC, you might guess by chance that there is an improvement with the CLC both times and conclude that it works. If you do the experiment a hundred times you will not consistently guess correctly that there is an improvement with the CLC (unless by some miracle the thing actually worked). You need to do the experiment many times, and consistently determine when the clock is in the house. If the effect is as profound as some people claim, you should be able to easily determine if it is in the house or not 100% of the time.

I applaud Zaikesman's efforts to bring some sanity to this, but for me the damage is done. I simply cannot trust anything that people write here nor anything published in the audiophile mags. I've heard good music in my system so I know there is something worthwhile in high end audio equipment, but audiophiles appear to be a sorry lot and are not doing much to help the hobby with all of this silliness.

Tbg: I'm still not following your point about sample size, but it sounds like you are saying that it is possible to manipulate statistical methods to prove anything one wants to by the way one chooses the sample size...?

If the effect of the CLC were profound, as some people claim, you could prove this with a very high level of confidence with a very small sample size - there is a 1 in a million probability that someone could correctly guess, purely by chance, whether the CLC was in the house in 21 tries.

A larger sample size is only required if the positive effect is very small, in which case you need a large sample size to show that the small benefit is statistically significant.

I expect that 10 tests would be enough to convince most people that this device does not have much benefit, and a larger number of test would only show with increasing certainty that it has no benefit at all.