There is a german psychologists study of 3 audio systems blind tested to more than 50 members of the public. Very interesting, the trouble is I can't remember which web site I read it on. Might be linked through audio asylum, or audionove.
Blind tests are complex, not just because some of the differences are percieved over time, and in ways which are difficult if not impossible to articulate.
As far as your scepticism of some refinements, I was too, but you can hear differences clearly if you exagerate. If you have biwireable speakers, wire the bass with ordinary shopping mall quality wire, and the highs with a decent audiograde. Then reverse the two. The difference should be easily heard. |
702 thanks for your clever and witty response. Nice to have you on board, though we often disagree. But that makes life interesting. |
Jadem6: Your imagination is running amok. I never said there could never be any difference in performance between pieces of audio gear.
Detlof: You're right, I'm no drummer. I'd be the engineer in the control room. ;) |
702..not so seriously intended but for the sake of a little laugh, which I hope you can share:
Who would you be in this your little story? The guitarist or the bass player...and p l e a s e don't say "the drummer". Regards, |
70242.241, How could there be an inverse relationship between price and proformance? I thought you just finished enlightening us on how there can be NO difference. Kind of proving yourself to be a bit two faced, huh. CHAH! |
Bob_bundus: Double-blind testing is useful only if you want to know what is truly audible. If your main interest is nameplates, model numbers, and prices, it's not useful because it might at times show a non-existant or even inverse relationship between price and performance. But rest assured, if you fear ever having to judge audio gear solely on how it sounds and not on knowing or being told beforehand what you're listening to, there's little chance you'll ever be put to that test.
But this controversy reminds me of this joke: The drummer stepped outside before the recording session and was surprised to see the guitar player outside already, but sobbing. "What's wrong?" the drummer asked. "Here we've got this important session to play, and the bass player detuned one of my strings!" the weeping guitarist answered. The drummer then asked, "So why don't you just tune it up again?" Protested the guitarist, "But he won't tell me which string it was!" |
Katharina said it best: "double-deaf listening tests". LOL Because that's how useless this testing methodology is, at least to us. |
Good switching will not affect the sound quality. That can be tested for by doing a double-blind test on the ABX switcher itself, by swapping it with straight wire to see if the listener can distinguish the difference. |
How much is the effect on the sound quality of the component under blind tested due to the addition of switching gear? You gotta have switching in order to test right away in between different tested components. |
Thanks for the laugh, Steve. Good one! |
The rodie for the Greatful Dead could actually see the distortion from the speakers. I know, some of you may scoff at this, but for those of us with highly developed audio abilities blind testing is clearly out of the question. |
PS: I will also assume that the cancer in the rats exposed to stock cable is in remission. Life on the Internet can be a positive addition to life in the real world after all. |
Good ending JD, I was deeply concerned about the future of the rats. |
"Not related to anyone who would claim me" Reminds me : /when I was 3or 4;our family moved. It took 3 or 4 months but I found them. Jaydem6, Who says we injoyed your post? About the recent 20 rats;we ALL believe you. "Coincidence? I truly think, (not)" This is actually 100xs better than "My speakers kill your speakers" Thanks,JD. Good ride!! |
Just a quick thank-you to all who have expressed there enjoyment of my post, it's nice to see we can have a little fun without taking our selves too seriously. I doubt too many of you will believe this, but the 20 rats exposed to designer cords were found in my lab this morning when I came in to clean up. It seems they let themselves in through the air conditioning vent and were able to assemble a very crude electronic synthesizer. The twenty of them were working as a team (orchestra of sorts) and producing the most incredible variation of Mozart's "First Movement (Allegro) from Sonata No.15 in C Major". Coincidence, Oh I truthfully think not! J.D.
Sorry Dekay, I'm not actually related to anyone who would claim me. |
George, shall I get some more, to make you feel even better? Glad to and thanks! |
Garfish:..... Have we progressed to Triple-blind-testing now?
Detlof: ......I neutralized the minuses/ was that a good thing? It was good for me.
Jadem6:...... Outstanding novel. Even Woddy Allen,Marty Scorse,James Cameron et al;have to cut / I'll wait till the movie comes out. Is that a coincidence? / I think not. |
I did some blind testing while shopping for a big screen tv recently,and i definitly thought the pioneer elite had suprior sound to the toshiba,but in the end i told the salesman the pictures were both toooooo dark. |
JD: Way hilarious post. My wife wanted to know if you are related to Dale of Dale's Dead Bug, or is the writing style just a coincidence? |
Just got four minuses more for my second last post. Someone must have taken it personally. Good! |
Jostler, we are of the same school here. The other day though, I changed cables between my cd player,the Purcell upsampler and the DAC, not really expecting anything in particular, perhaps a slightly better definition and coherence and to my surprise I seemed to get a wider soundstage. Tried all sorts of software and the phenomenon did not go away. Changed back to the old cables and there was shrinkage. Cannot explain this. Mabe its really a case of those solder joints! But then jitter would not influence soudstage, would it? |
Detlof: Absolutely, for something like soundstage/imaging you need a complex stereo signal. (What would stereo pink noise soound like??) But I'm from the school that says soundstage is primarily a function of the source material and speaker/room interaction. The sorts of things you would do blind tests on aren't much of a factor (unless there's a channel imbalance somewhere). When people claim that a cable gives them a wider soundstage, I get skeptical. |
Oops, just got my self I nice set of minus points. 4in all.
JD you were right, if you try to be funny on Audiogon, you get into trouble. You live and learn. What I dislike is the anonimiy in it all. They can crawl out of the woodwork and then snuggle back in. Aw well. There is freedom of speach ain't it and even the greatest knucklehead is entitled to his opinion. So I will not retaliate in kind. On the contrary, I aplogise if I hurt anybody's feelings or sense of propriety , but then she/he should speak up.
Jostler, thanks for your reply. I think the people tested were really deaf at these frequences. That had been established beforehand. Did not know about the pink noise experiment. Intuitively it makes sense at first sight, but would you not think, that more complex things like soundstage spread in all its possible dimensions would need more complex information than pink noise, which by the way is not without complexity in itself. ? (Still think this is a great thread, even if the minuses should keep on coming!) |
JD........one more thought-- Maybe you could enlist the help of the "three blind mice"......put it all to music, and you could have a classic blind test of whatever you want. |
JD; incredible post. Nice beat, easy to dance to, I'd give it a 9.9 and I voted accordingly-- heh heh heh. :>) Your post makes much more sense to me than not having enough confidence to trust in your own senses. Is it psychological? You bet. Are you funny? You bet. Is humor psychological? You bet. Can humor be measured? Only psychologically. Maybe you could re-capture some of your rats and test this. Cheers. Craig |
Detlof: Dont' know about studies of the hearing-impaired. I can think of a number of plausible explanations, but I'd be speculating (as opposed to listening) blindly. For example, could they not hear those frequencies at all, or was it just at a lower level? (In other words, how steep was their low-pass filter?!) Could be they heard enough to detect something, but that's just a wild guess.
As for your mathematical model, it would run counter to experimental evidence. I know of one test of cables where listeners could distinguish between two cables using pink noise, but not using a piece of choral music. That probably won't surprise too many audiophiles. |
JD you sound like a 10 hz tone to me...you have me stand in awe, not only do you keep late night studies with the ladies, you also do the most complex experiments in other fields until, what was it, you had two of the 24 year old ladies squished? Terrible. You must be Dr. Strangelove in real life, please admit it. There is so much of that hz tone in my ears, that I am AWEfully confused. What will you do now...get new ladies and fiddle with their shortchanged abdullas? WOW... and Jostler, thanks for the enlightenment. I would say psychoachoustics is even older, it goes back to the Greeks and early China and India, but you would say, that was prescientific, which of course it was in a more narrow sense. You were right in assuming, that I meant the emotional impact of music on us, where to my knowledge science so far has no real answers yet.A question, seriously, don't want to bug you, I had put it in another thread and never got an answer: Do you have any idea for an explanation about those odd experimental data, which showed, that if you play music to people whose hearing is limitid in the upper ranges, I forgot where the cutoff point was exactly,but it must have been around 10 khz or above, and in playing it, you cut off the frequencies, which they were not able to HEAR, they all and sunder at once noticed this fact all the same practically always and could thus differentiate between music played normally and cut off in the highs, even though "psychoaccoustically" they should not have been able to hear any difference at all. Another thought, I wonder if you could not develop a mathematical model, which would "prove", that the more complex and "rich" a musical offering, the greater the chances to hear "subtle differences". Would be an interesting hypothesis to go after. JD bring out your 24 year old rats, or what was it again, the 24 girls from P.E.T.A? I am so confused, there is that tone again........... |
Detlof: Psychoacoustics is about a century old now, which is pretty long in the tooth for an infant. And it's not about music. It's about perception of sound, of which music is a complex example. (How music moves us emotionally is another field entirely, but if you can't hear it--or feel it, in the case of that pipe organ--it isn't going to move you.) Also, the complexity of musical sounds actually makes it harder to hear subtle differences, not easier. People generally score higher on blind tests using test tones. So those "limited parameters" actually give us an upper bound for what is audible using a musical source. |
I am sooooo mad! I've held back from all of you on this blind testing thing, and now... Geeze I'm mad!
I've been running a controlled lab test using 150 non-related white rats from all over the world. I've had each one implanted with an electrode at the stem of there brain to monitor there reaction to sound. Of the 150 total, 20 were set as a control group and only listened to standard power cords. The rest had varying periods with standard cords and periods with differing "designer cords" I was 5 1/5 months into a six month study when it appears that either 70242.241 or P.E.T.A. broke into my lab, broke my computers and released all the rats. This of course is not only the end of my study, but sadly the end of the rats. The implants do not allow these rats to hear anything unless they are plugged in to there headphones. So sadly they are perfect targets for cats, birds and cars. I found two squished outside my lab this morning. A full investigation will be conducted to find these audio haters. Luck for all of us, all my records were backed up and in a safe here at home. I can only share preliminary findings at this time, but the results are startling.
First glaring fact, and I might add a major concern has to do with my control group. It seems that of the 20 rats exposed only to standard power cords, 15 of them (that's 75% for you scientists) have a rare form of inner ear cancer! Coincidence, I think not. This alone is worth the value of the study, but some other facts can also be gleamed from this preliminary information. It was found that in 90% of the cases the rats had lower heart rates and greater brain activities when exposed to designer cords. Conversely while rats were exposed to standard power cords the brain waves were identical to Jeffry Dommer. Coincidence I think not. Now each rat was exposed to differing amounts of time with designer cords, and some very interesting facts came to light. There was a direct correlation between the amount of time on a designer cord and there I.Q. along with there taste in music. It seems that the control group were only settled when top 40's pop was played. The next level group (20% designer cord) were only content when listening to disco and country. On the other end of the spectrum, the group exposed to 80% designer cords preferred classical and jazz, but not any jazz, Coltrane and Davis it seems were there preferred choices. The group with 100% exposure seem to show a very wide array or musical preference, but oddly they only seemed truely content with high quality recordings. Coincidence, I think not. Now I had also discussed the variance in I.Q. This study was not intended to measure an I.Q. to power cord relationship, so all this is subject to further study. We did have a series of rudimentary tasks we asked each rat to preform, and while not conclusive it was of interest. The control group was able to successfully complete only 15% of the tasks while the group exposed to 60% or more designer cord were completing the tasks with 100% success. Now this in it's self is not indicative of I.Q. alone, but could also be explained with there level of contentment. The startling fact was that the group exposed to 100% designer cords would ONLY except the Wall Street Journal for the bottom of there cage. Coincidence, I think not.
Sadly for all who are interested in the "advancement of high quality audio" A.O.H.Q.A. due to the alarmingly high percentage of cancer cases in the control group, the entire study must be sent to The National Cancer Research Facility for further review. So the results will not be made public until a conclusion can be arrived at regarding the cancer issues.
In the mean time I am starting a new group to combat the sick people of P.E.T.A. I'm calling it P.E.T.A.F.F. (People eating tasty animals for fun) If your interested in joining just let me know.
J.D. |
Sorry, not JD, Jostler is the scientist i meant.....and ahem Jostler:
1. You cannot train gullibility. What you can train is to be a better charlatan in knowingly to pretend to hear differences, which are not there or to pretend not to hear differences which are there. You cannot train something which is preconscious--like gullibility, suggestibility, naivite and such.
2. Scientists who study hearing do so in controlled circumstances which are generally carefully limited to a few essential parameters. All this has nothing to do with the complexity of a musical event, be it live or in audio. The science of psychoaccoustics is still completely in its infancy. So far there are no satisfying explantations why music does to us what it does, except for a few isolated facts, like that frequencies below 16hz or so, will scare the shit out of us. Will induce awe. (Used in the church with a decent organ and by the ancients in battle) |
Oh heck JD, I forgot France as well. My Jadis stuff just hates it, when I click my heels. All the tubes flare up red in a very dangerous fashion and then there is a bubbly sound through my Quads, like that of VICHY water being poured. Maye I should be kissed by a princess and be turned into a frog and then klick my heels. that might work for the Jadis. But I don't know any royalty.....and I don't know how frogs hear. Jademo you are the scientist, how do frogs hear? |
Yes JD, you make sense, Holland is close to Germany, but then you should only klick heels, wearing those funny wooden sandals, the Dutch are famous for and that's also why all the SACD stuff needs wooden cones...and besides stereo, did you also try multi channel and did you use one or serveral power purifiers at the time? I am sure your powercables were Master Couplers, weren't they, but then probably Jostler the wizard will prove me wrong. But all the same I would like to ask him, if one or more ladies were inadvertedly to sit on the gear fired on, would that cause a brown out? Oh and don't worry JD, I won't play with positions at all. I've got my speakers on cones you see. |
I'm lost now, I've got USA, Italy, France and Japan going. I guess I can still click in that Philips helped develope my SACD! Detlof, My experiments went on for days, it's tough keeping six women entertained, oh and we had the stereo thing too. Lights off is good to start, you'ld hate to scare them all away. When morning arrives they just want more, more power that is. That's when you break out the big one, ah the Krell, yea that's it the Krell. Good luck my friend, pace your self and don't try all the positions, ah for speakers I mean the first night. |
A little story comes to mind:
When CD's first came out, there were those nuthead believers, who maintained that interconnects between the CD players and the other gear sounded different and worse, that they even had directivity. Of course they were ridiculed by "Science" and probably told to klick their heels (-; ( by the way,works only with gear from Deutschland) because bits were just bits after all. Until it was found out, that even minute faults in the solder joints would cause jitter and suddenly those first ridiculed, became rehabilitated. The history of science is full of similar stories. It is not just a question between knowing and believing, there is something inbetween, namely "experiencing", gathering empirical evidence, which would lead to new hypotheses ( and those at first are generally "beliefs" ) which then in time could be turned into knowledge, widening the scope of science.
I think we should not bicker with each other, but rather listen to and learn from another. The believers should learn to be sceptical and careful, the scientists to be curious and openminded. Obviously looking at many posts here in Audiogon, this is a difficult thing to achieve. Everybody rides his own high horse, he should rather get down of.
Well, I'll get off my donkey now too and shall try to repeat J.D.'s experiment and invite six 24 year old women over and then we'll see (hear) about them cones, hopefully , I'll not be too distracted. JD, did you listen with the lights on or off? This seems an important parameter as well. I've got goospimples all over in nervous anticipation and I must be off now and get on the phone, to get things rolling. Sorry Jademo, can't klick my heels, because my gear is either Brits or US colonial. Should I sing "Rule Britannia" perhaps? |
Jostler,I enjoyed your wizardry,and voted it as such. Now what want to know; should I be blindfolded,when I click my heels? Or do I also need to do the 6 women thing? Does that need to be done blindfolded also?Does the blindfold go on them? According to all my many "friend"---Yes |
Couldn't agree more, J.D.--this is about belief, not science. Just click your heels together, and your system will sound great! |
Well Detlof, I can not begin to thank-you for your insites. I had to finally ask the six 24 year old women to leave. We sure learned alot. Oh and we listened to the system too, the verdict, there are those who believe and those who don't. Of the ones who believe a group of them have amazing sounding systems and are not concerned with the viewpoint of those who don't believe. If those who don't believe want to stand on there "scientific" belief great, too bad for them for there lose. If you've followed the threads of the na-sayers above I think you'll find there experiences are few and there view points limited. It's clearly not worth the energy to try to help these people find great sound, they are not here for that. They only want to prove there great knowledge and little practical experience. The best test, six women, mid twenties, thanks buddy! J.D. |
Detlof: 1) The power of a trained ear depends on how it was trained, and for what. If it was trained by years of imagining differences that one couldn't really detect, then it's not much use to anyone else. The refusal of many subjective reviewers to submit to objective testing speaks volumes about the quality of their "training." 2) Scientists who study hearing will tell you that they can measure differences far smaller than the human ear can pick out. 3) I've no interest in getting into an argument with you about your blind tests of cones under preamps. But I'm very skeptical. It would not be a trivial exercise. |
Mike, I think you make a very good point here, namely that we are all prone to be taken in by clever marketing and gushing reviewers, however I think you underestimate the power of a trained ear. I think that well trained hearing acuity beats "modern technology" any day in hearing subtle differences. On the other hand, I'll grant you the case of Vladimir Sushurin, the designer of the LAMM gear. He seems not to listen at all, only to "measure", but then measure according to his very special theory of human hearing, which, if I am informed correctly, runs very much diverse from mainstream thinking on this topic. I myself can hear the difference cones make, not under all and every gear, but under some, CD players especially and some preamps, yes even in blind testing. I think its difficult to generalise in this our topic. There are lots of opinions floating around, but no hard and fast rules, except for those basic ones, which however do not necessarily indicate exactly how a given unit will perform. Not that it matters, on the contrary, it makes life more interesting for all of us. |
FACT! If you were to take all of the major audio magazine editors and reviewers, you know, the ones you trust your thousands of dollars with, and had them BLIND test every component (excluding speakers)From the cheapest to the most insanely expensive, and then had them write a full review on each one the results would be shocking!! The reason they don't do this is because (1) they would certainly lose business from the big players and (2) they would emberrass the sh** out of themselves. Personaly, I think that a professional reviewer that claims he can hear the difference after installing cones on the underside of his amp is full of crap! What's even funnier is that this same reviewer claims to hear suttle differences after he turns the cones up-side-down!!! Please, folks, get real. If modern technology cannot detect a sonic difference, neither can you. If you want to save yourself thousands of dollars on audio equipment, try LISTENING without knowing the brand name, model, or price. You may feel like a fool but believe me, the salesman will respect you for it. |
Jostler3, the ABX page is no longer on the URL you cited. It's here: http://www.oakland.edu/~djcarlst/abx.htm
It's got very good documentation of the double-blind test process and explanations of the statistical analysis involved in evaluating the results.
Also, in a white paper by Dr. Floyd Toole there's a description of tests where the listeners were allowed to see the speakers they were evaluating and tests where they were not; not surprisingly, in the "non-blind" tests they gave higher marks to the speaker systems that looked more attractive. The white paper is at http://www.harman.com/wp/pdf/AudioScience.pdf.
I like double blind tests. It's fun to take them and to witness others taking them. Especially when there's an "everybody knows a Goldbrick M9000 sounds better than a Conethumper Gizmotron" type of comparison being made. Well, after the test, maybe everybody doesn't know that now. Or maybe a listener will hear audible differences between A and B and find that the one that sounds better actually isn't the one they assumed was superior.
I believe in getting value for what I spend on audio gear. If something commands a premium price, it ought to offer comparable audible benefits. And I stress again, audible. If not, keep it, because I'm not interested in impressing people with how much I can spend. I just want great sound. |
Hi Don, I think your original thread is a very complex question to start with.
Audio Engineering Society 2000 November journal published an excellent paper written by Seymour Shlien, "Auditory Models for Gifted Listeners". The author introduced the subject about certain professions like musician, piano tuner, recording/mastering engineers whose job depend on their better than average hearing ability. What's significant to this Audiogon thread is that his test results on 14 subjects identified some outstanding listeners in having (if I may quote) ".. unusual ability to hear signals or discriminate small differences. This skill show itself among audiophiles or musicians who may be willing to spend significant resources in order to achieve the best audio reproduction quality. Though these listeners are hard to please, they are an important component of the audio consumer market, and some effort should be made to understand the reason for their special needs." The paper has real tables & charts, very scientific...
As an audiophile and AES member myself, after reading this paper i felt reassured and understand my hearing capabilty and needs better now. Don, you should also try to get hold of this AES issue. Regards Phil |
|
As a medical man, and a proponent of "evidence based medicine," (which means therapeutic decisions should be based on clear evidence of efficacy vs. fads or favorites,) I think there is a very real place for double blind testing anytime one wants to make definitive statements on performance. None of us are immune to the esthetics of our components, (unless we are blind,) nor are we unswayed by favorite companies whose founders, representatives, or dealers we are fond of. As I read the original post, the question was to blind testing as a part of the review process, which I believe to be legitimate. I would love to watch the hotshots at the mags in a double blind test. I think we might be very surprised at the outcomes. That does not mean that any of us should enjoy a piece less if it does not stand to blind testing critique. I, for one, just like the way some things look on the rack and it adds greatly to my enjoyment and the whole experience of music, so I will keep them untilÂ…
Charlie |
Dtittle; I think in my first response I spelled your name wrong-- sorry, and I looked closer this time. Thanks for your response-- the equipment you have certainly indicates that you are "one of us". I mentioned critical listening skills-- well, they could be very effectively used in blind testing as well as sighted. Avguy, JD you guys crack me up (and I voted accordingly), and I hope humor is never disallowed on Agon. Actually, I flunked my last test Avguy-- too much sugar:>) It's nice to have a polite exchange on this contentious subject. Cheers. Craig. |
Mr drubin: I guess "both",but who wants their shortcomings pointed out to them?(nobody I know) Mr Dittli,yes the mags like to tweak us;and yes we must listen for ourselves/with our own equipment.This is not a firing squad/ must we be blindfolded? |
Don: First of all, there is a difference between a "blind test" and an "objective test." All objective tests are blind (double-blind, actually), but not all blind tests are objective. It's still possible to be fooled into thinking a difference exists when you're listening blind. As for references, check out the "official" ABX page:
http://oeonline.com/~dcarlst/abx.htm
It contains background info, plus a list of articles that discuss or use ABX testing.
Your skepticism is quite justified. The primary reason magazines do not do objective testing is that it would leave them with too little to write about. Besides, it would really tick off a whole bunch of advertisers. The exception, of course, is The Audio Critic, which you should also check out. |
Is it insecurity to need a blind test or is it insecurity to reject or fear such a test? |
The following article addresses the subject of double blind listening as it applies to high-end audio:
C.J. Huss, J. Gordon Holt, Larry Archibald, et.al., "The Highs & Lows of Double-Blind Testing", Stereophile Vol.8 No.5 (May, 1985).
It can be found on the Stereophile web site in the archives section. |
Having started this I would like to jump back in and respond to a few of the above posts. First, I am not trying to imply that their are not clear differences between many products. However, since many of us do not have the opportunity, due to time or money constraints, to conduct our own blind tests of the products we are considering we must rely on the reviews of others, particularly the audio press.Given this fact, I want some assurance that the claims being made can,at least to some degree, be proven(and yes I realize music is inherantly subjective).So,again, why not make blind testing part of the process. Isn't more info always better than less??
As for why I want to see some actual studies done on this subject, its because there is a part of me that wonders if all of this isn't part of a giant marketing scam. I hope not, but I'm still curious. The next time you read an audio review take a step back and look at what is actually being said. Words like "timbre" "bloom" "etchiness" "open" etc,etc. are easy to throw around, but if someone is not willing to submit to listening tests then my guess is it is because they might end up looking foolish.
In response to Garfish,(whose posts I always find informative) I have Krell electronics, B&W 804's, audioquest interconnects and cables.
I look forward to all comments! |
