Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)
About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.
Yes, each meteor produced a sizzling sound, like the sound of a frying pan.
Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.
ASR would have agreed with this sound reasoning based in elementary science.
Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.
Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.
I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.
| Post removed |
@prof : I would appreciate it if you answered my few questions. If you forgot them , just ask me. I understand |
You CLEARLY read each of my posts in this thread and follow up asking me questions directed at what I’d written. But then tell me "you don’t read my posts." Do you imagine you are fooling anyone at all?
Given you were lecturing Amir on decorum, I think you fit that definition quite well. One could add "liar" (for claiming not to read my posts when you clearly do - in fact you are still even QUOTING my posts!). And of course: Troll. Collect ’em all....
|
@prof :
Vs, Devore speakers? That thread you had 78,000+ views on? Based on ABX blind test between the two I presume? If so, please do what you asked another member here:
|
@prof : I thought you bid me farewell’s many posts ago? No? It hurts? P.s. I don’t read your posts. I know exactly what you would say. I am not a hypocrite. Kindest regards |
You aren't trying to understand. You glossed right over the fact I already gave the answer. But for anyone else reading: As I said, the difference is: the proposition that different speaker designs will be quite audible is not only uncontroversial: it is well established in practice, theory and also via classic scientific methods (e.g. blind testing of speakers). That is NOT the case for many of the more controversial claims in audio, such as that expensive USB cables will sound "better" than properly functioning cheap USB cables, or similar claims. Therefore it makes sense to treat the more controversial claims more skeptically. Of course such obvious differences will be hard to spot if someone is wearing black-and-white blinders and can only see in an "either/or" manner. |
@prof : why do you selectively quote only a small portion of my answers? It reminds of the doctored Big Lie stuff. You are certainly learning from Putin And why you don’t answer my questions? Inconvenient for you? |
Of course you didn't. Your trollish behavior was obvious in every post. Now, I'm not sure what some people get out of such behavior on forums, but...hey...if you choose to keep sitting yourself in the dunking cage to throw out ham-fisted "critiques" and attempts to insult...don't be surprised if you keep getting dunked ;-)
|
@prof :
So, you admit this, but, at the same time, you questioned someone’s findings without posting proof of blind test? I call this hypocrisy. Please correct me if I misunderstood
prof3,276 posts @ other Anecdotes are not that evidence BTW, you say you’ve distinguished all sorts of things in blind tests, including USB and AC cables? Can you describe your test protocol?
|
@prof I will not pretend I read your reply. I did not. Unlike you, I will not be a hypocrite.
‘’My question is: I thought you bid me farewell? No? It hurts? |
^^^^ This from the guy trying to "tut tut" Amir for being insulting.
Grow up.
If you bothered to read my posts that I linked to you’d know the answer: No blind test. All sighted conditions. Which means I happily admit some forms of bias may be playing part in my impressions. But I’ve defended why I’m fine with that many times on ASR. Unlike, say, the case of super high priced USB cables, it is not technically controversial that different speaker designs sound different. In fact it is well documented in theory and practice. So, while sighted bias can always play a roll, it is not at all implausible to be hearing distinct character differences between different speaker designs. If you think you have found some "gotcha" (which is all you care about) you haven’t a clue. If you find this confusing or seemingly inconsistent, it’s because you see things as "either/or" and don’t care to understand a nuanced position where both objective and subjectivity play a roll. So, check your own dogmatism.
|
What does that have to do with what I asked. A “member “ can send you a bunch of stuff for various reasons. Does that make you feel happy and accomplished with your life?
Also, if he spent so much money on a cable (assuming not a $20 Ali fake), don’t you think he is NOT your audience?
|
@amir_asr great and dandy but why anyone would pay thousands of dollars for a Mola Mola DAC if a $50 CCP DAC can measure just as good within humans’ audible thresholds of hearing? Aesthetics? Explain this to me.
Do all Competent DACs sound the same? Amps? Preamps?
Thanks! |
| Post removed |
Answering for him :), you look at measurements. If frequency response is flat and independent of load, has distortion and noise below threshold of hearing, then you can very confidently declare is transparent. This analysis assumes perfect speakers. To the extent the speaker is not, then the job gets easier and hence the reason amps with noise and distortion above threshold of hearing are also declared as transparent. If anyone challenges you saying they hear the difference on their speakers/headphones, then you can perform a level matched blind testing test. if they can't prove any audible difference in such a test, then the case is closed with respect to their claims. |
If I were a Chinese, I would find this very insulting. And dare I say, racist. But I am not Chinese, but are you? If you are of Chinese descent, even if one of your one of your great great parents, you are absolutely excused
@amir_asr please change the wording before time runs out. I am very curious, what happened with your initial measurements of Vinnie Rossi linear power supply and his subsequent success later on with his power amps and highly priced integrated amps? He “fixed” it?
|
Take this from a different standpoint. If speakers and headphones are so lacking in transparency, how could you ever differentiate two amplifiers that are much more transparent than what they are connected to. Looking at it another way, two amplifiers must be significantly different such that they can be differentiated when connected to a typical loudspeaker.
|
Consider it? That is like asking a Chinese person if he would consider eating rice! My measurements get verified by manufacturers every day of the week and twice on Sunday. You think these companies just lay low and ignore what I am producing? Countless companies have asked for my Audio Precision project files and I happily provide it to them so that they can make the same measurements. To the extent I have a relationship with a company, I even offer the measurements for them to verify prior to publication. Here are some quick examples staring with a German company you should know: Neumann: "EDIT: received an overlay of Neumman measurements and mine: Here is Denon:
"I grabbed a preliminary set of measurements from the DAC section of the 3800H and ran it by the company. Within typical margin of error, the measurements were the same as company's own." Here is American company RSL:
"I was contacting the company regarding another product so while I was at it, I asked them to review the frequency response measurements. They are generally in agreement with the results you are about to see." We also have heavyweight reviewers and engineers with their own measurement gear verifying my work. Here is Seymore who writes for AudioExpress: "If there’s anything in his data that you think he’s hiding, then request the AP project files. When I have, Amir has cheerfully provided them." Seymore has also tested some of the same units I have with same results. In this case the designer of a phono stage claimed my measurements were wrong so with permission of the owner, I sent the unit to SIY (Seymore): "As you can see by comparison, our measurements correlate quite closely (though we use opposite numbering for our channels, which I'll blame on my Hebrew education, learning to write from right to left). So I can proceed without worrying about major inconsistencies." It is also not hard to replicate my measurements after seeing them. Such is the case with Professor "Wolf/L7" in China where he tests domestic products that I also happen to test. Results are almost always in agreement. I also helped another reviewer (Erin) who had gotten a Klippel NFS measurement system to get up to speed by sending him the very sample I had tested (JBL monitor). Really, there is nothing to hide here. I put out these measurements with full expectation that they will be scrutinized both on ASR and by manufacturers and other testers. If you have a third-party in mind to verify my results, I am happy to send them one of my tested samples for them to re-test. You pick the place, make it worth their while to do the testing and I will do the rest. I suggest reading ASR for a while and to better understand what we do as to not ask questions for which there are trivial answers. |
@prof : don’t be shocked. We both know what I mean. Are you still in good terms with master? All the good work you are doing here must pay dividends and must not go unnoticed by master 😉 ‘Oh:
please describe your test protocols you conducted in choosing your current speakers vs. the Devore speakers you must surely conducted when choosing your current speakers
|
Huh? No I ended up choosing Joseph Audio Perspective speakers. Still love the heck out of the Devore sound, but wasn't ultimately a good fit for my room. What possible point do you think your question makes?
What in the world are you on about? You just asked for an example of a subjective oriented thread in which I participate...I give you a WHOPPER of just such a thread, and now you are dancing around to some weird conspiracy theory or something? What...I've spent the last 5 years on that thread exchanging only subjective impressions of gear...but it was all in the service of a cover for my Secret Identity in which I don't REALLY care about the subjective impressions that fill that thread? You are clearly not here for a meaningful or serious exchange. So long...
|
@prof : so, you mean you own the Devore speakers? And I am very curious: why do you pretend to be something / someone you are not? Everyone knows where you belong in the Great Debate. Do you have the testicular fortitude to say it? there is no shame. Let’s say it together: Amir. is. my. master.God.Holiness |
First: please interact like an adult. Thanks.
You don't come off as "genuinely" interested in your post to me, but it is curious that you've been here a long time yet seem unfamiliar with my many posts describing the sound of equipment. I have for instance a years long on-going thread - currently almost 76,000 views! - in which I report my subjective impressions of many speakers as I audition them:
|
I appreciate the philosophical spirit in which you made your original post and have presented your view in the thread. However, the reasoning in your OP is very problematic. It is a version of the "They called Galileo crazy" argument. "They" (in the modern reference, now it’s science) thought something wasn’t possible but "it turned out they were proven wrong!" Yes, that is true sometimes. But you should see the issue immediately: HOW was anyone proven correct? By....The Science! Right? In other words, by strong, rigorous methods of empirical inquiry! That is how we weed out the justified claims from all the competing ones! There are countless ideas that people believe, but the only rational way to sift through which ones are likely or true is to wait for the strong evidence any claim demands! Remember that for literally EVERY crazy idea anyone has ever proposed, every bit of pseudoscience, every spiritual claim, every wild "cure," claim you can find at your run of the mill New Age Festival, you will always find people saying "You shouldn’t be so quick to dismiss this, science hasn’t explained EVERYTHING you know, and people like Galileo were thought crazy at first too!!!!" It is THE go-to hand-wave in defense of bunkum. And that should be a huge red flag for you. So the problem isn't that you are necessarily wrong in what you may happen to believe; it's that your reasoning can be used "in defense" of literally anything, and hence it’s not much of a reason to help justifying believing anything in particular. Since people can make up any claim they like, and also people can imagine all sorts of things, when faced with a dubious claim, for instance a technically controversial claim in audio, the rational stance isn’t to say "so it’s ok to believe this because who knows if it will be found true via science in the future." It’s to WAIT for science (or engineering) to sift THAT claim out of the countless other competing claims, as actually being true or justified. And also, if CURRENT electrical and psycho acoustic theory and practice points to a claim being highly unlikely, then it makes sense to want some rigorous evidence in favor of that claim. Anecdotes are not that evidence. BTW, you say you’ve distinguished all sorts of things in blind tests, including USB and AC cables? Can you describe your test protocol?
|
@prof : Matt Hooper. I am curious, with all the defending and Good Samaritan ASR work you do here, did Master Amir and his lieutenants forgive you for what you did on ARS? Just curious. He loves you again? No broken heart bad feelings?
For the below: please point to one single “subjective aspect of audio” thread you still enjoy and participate in. I am genuinely curious. In my humble opinion, all Bs. You know where you belong. What you say is just pure BS, to make people see you seriously for what you are NOT. See your subsequent posts below I am sure are coming, more Catholics than the Pope (Read more Amir than Amir)
|
If one thing is now predictable in the audiophile world, it's that any discussion of ASR in forums like this will result in reams of misunderstandings, misrepresentations. I mean..just...wow...this thread is like a straw man menagerie! I've been a member of Agon for a long time and still enjoy this place for discussing the subjective aspects of audio. And I'm often defending subjective impressions and reports on ASR. But threads like this remind me of why I'm so glad ASR also exists.
|
I am 100% sure it’s Cin Dyment. Wait until he creates a brand new username here, and posts a bunch of screenshots of my posts. 😂😉🤦♂️, on this very thread.
|
@thyname If speakerdude actually is atdavid, he has modified his behavior enough to disguise his true identity(s). Behavior modification is subjective. Even he would have great difficult in measuring this modification objectively. LOL. There are shared similarities of those former usernames: - Dislike of audiophiles. - An obsessive need to post on audiophile websites. - A need to demonstrate how smart he is. - Constant examples of how clueless he is about audio. |
@thespeakerdude how would you know if say an amplifier is audibly transparent in a listening tests when listening through speakers or headphones when no headphones or speakers are transparent? |
No, both @invalid . But a listening test, not just sitting there and saying it's different. Repeating what I said before. I can play the exact thing twice and many audiophiles will claim they are different. Audiophiles like to rage against blind testing but it's the only way possible to remove bias. |
Oh Cin Dyment @thespeakerdude : I thought you were limited to that one thread. But just like your previous 17+ usernames here (see list of your prior usernames below) you are proliferating everywhere, like you always have done. This time, slowly but surely. Sad bad creature…. ———- thespeakerdude
cindyment2
oddioboy
crymeanaudioriver
theaudiomaniac
theaudioamp
deludedaudiophile
thynamesinnervoice
cindyment
snratio
yesiamjohn
sugabooger
dletch2
audio2design
dannad
roberttdid
roberttcan
heaudio123
audiozenology
atdavid ———————
This time around you are pretending to be a speaker manufacturer. Which we both know it’s a made up fantasy. Post your credentials here, or even a list of your current stereo equipment (no need to post pictures or proof of what you play music with, I will take your word), and I will eat crow, no matter how many times you hate audiophiles here |
I think ASR is a great service to community. Here is why:
Imagine a young dude with no money still in parents’ payroll. How would he base the decision on what to buy? He has no idea what good sound is, and no local audio stores. And he has no money, remember.
That’s where ASR comes in play. Bingo! Cheap China stuff that measures best. Numbers don’t lie. Parents HAVE to open their wallet based on proof from a highly regarded scientist. Beat it! |
Or @andy2 , looks like the vendor got caught making claims that were not all true and a "feature" that was not a feature at all and is now doing damage control. |
There’s a little more to it than that. For example, the logic and premise are examined, and that step alone would disqualify a lot of the "science" as seemingly practiced at ASR.
That is absolutely true.
Yes, both extremes exist: papers that have little or no value, and papers that have enormous value. What really happened to Pons & Fleischmann in the early ’80s (hint: the University of Utah really effed it up) explains what can happen when science goes wrong, even when respected researchers are involved. |
@thespeakerdude audibly transparent according to measurements maybe, but not in listening tests. |
A lot of scientific work revealed to the public is not peer reviewed and all that is reviewed is method. The peer review does not consist of other people replicating the work as you have suggested. You are moving the goal posts. Even with peer review, many poor papers are published and if you are presenting new concepts there may not be any one "better than you". On correlation between ad hoc consumer reports with little in the way of comparative analysis and measurements, there are many flaws in that statement. The most obvious being what is liked and what is accurate are often different if not almost always different. The other obvious reason is simply that people often assign better to new purchases, whether actually better, or in audio even noticeably different. Many people played the exact same thing twice will be absolutely convinced one is different. Either way, both of these are psychological effects of which measurements are immune. I would also say using the word "mass" is exaggerated, but if you want to go down that path, there seems to be mass like of some of the inexpensive gear they have measured. Audibly transparent is not a catch phrase and needs no further words or explanation. It is a simple concept. When certain products achieve a certain level of technical performance they are audibly transparent. You could make a product that measures better, but it will sound exactly the same as the one that does not. That is confirmed by the inability of people to hear a difference. You made a comment about "quality" of parts, but this would be a prime example. An expensive part is often that, an expensive part but offers no benefit in improved sound. The change would be transparent. No audible difference. |
Quite so. Typically, a paper would undergo at least some informal peer review before it is even submitted for publication. When it is submitted, it’s then subject to a formal review before even being accepted for publication. When that part of the process is poorly done if often reflects poorly on the publication itself, so although mistakes happen, it’s exceptional. There are those who wrap themselves in science, then use the pick-and-choose smorgasbord approach in practice. You see some do this with religion, too. Measurementalism is rather like religion in that it is taken on faith (there’s no need to listen!) rather than facts. Of course other factors influence these YouTubers, too, as has been noted. |
You say that publishing is normally the audit process in the scientific world. Scientific papers are subject to peer review. Publishing the results instantly is not a great idea. There is always someone more qualified to do a job. And that person ends up auditing and making sure the work is suitable for release. Audibly transparent is a catch-phrase I hear all too often. More descriptive language is required...such as using: Sounds Like? An Audio Glossary J. Gordon Holt | Furthermore, large discrepancies between mass customer reviews and measurements still exist with plenty of gear reviewed on ASR. You can find an audio product that has plenty of positive reviews, mentions particular sonic traits, etc. from numerous online stores (not just amazon). and even impressions from audiophiles in person whose experience lined up with yours almost exactly. Then you go to ASR, only to find that product does not score well there. And yet, we find a mountain of information from others that suggests otherwise...actually the total opposite. I think it would be great to hear from @amir_asr about 3rd party testing his measurements/results...let the man speak. @clearthink I agree 100%. The experience is all-important vs measurements alone.
|
Here is something so obvious, true, and valid that I am astounded and amazed that anyone would dispute it but of course those who have an ego to protect or snake oil to sell will rise to their feet immediately to protest usually with illogical personal attacks: If you want to review a car, you must drive it. If you want to review a movie, you must watch it. If you want to review an audio component, you must listen to it. If you do not listen to it, you can not be sure how it sounds even if you have measured it. That is because you have not listened to it and how any one can dispute, argue, or oppose that truth only reveals, displays and demonstrates a fundamental, basic, essential ignorance of the premise of science, or has been blinded by greed and/or conceit |
@mastering92 can you name any other review site that provides measurements that does this "auditing"? Manufacturers always have the ability to respond and provide high quality measurements to refute the results including testing methodology. In the scientific world, publishing is normally the audit process. You publish and encourage others to replicate your results. Contrary to your assumptions I don't participate on ASR though I do read, mainly speaker reviews. As I wrote above ASR publishes reviews from other people. I also see others publishing their results measuring the same products ASR does though obviously a limited subset. ASR is obviously not perfect, but all the arguments I am seeing are sour grapes. |
+1 @nyev such an excellent post. @mastering92 the ASR community repeatedly states that they are "right" because of science. You’re simply encouraging replication by other parties to verify Amir’s measurements and results. I’m sure logistics and extra work between Amir and other parties will be the excuse, but then honestly, unsure if the "S" belongs between the "A" and the "R." |
ASR does use a 32 tone test - or something like that...which is better than just 1 KHz. My post about your graphs (the bar graph) with different types of listners in a selective order was removed by a moderator! I’ve reached out to her a few times regarding site issues that have since been fixed! Was this done in error? Please consider that the content of my post tried to get the crux of an issue regarding ASR measurements. Anyway, I can post some of it again. Wish I would have saved it...wasted my time writing all that out. I’m not here to argue with you, but since you are here and alive/well, I figured that sharing some of what I believe is right is worth a shot. Audio _ Science _ Review Like I said before, Would you consider having your test results/measurements verified by other industry experts? I don’t see a problem with that. If I were you, I would welcome 3rd party validation of my work. Why? because that is a scientific process. I have a few friends who are actually scientists, work in labs etc. And we’ve spoken about this process many times before. If I’m actively posting reviews (like you) I would certainly want others to credit my work as correct, with repeatble results. Perhaps an online petition could be made public; then have audiophiles from this forum, your forum, and others shoot their shot. Either you get audited, or you don’t. It’s that simple. Would you be open to such a challenge? If no one it auditing your work (much like at a workplace within a finance department); we can’t just assume that junior accountant is doing flawless work. A CPA would be a watchful eye, so to speak. I think ASR needs one as well. Nothing against you or the folks on your site.or even @kenjit Let’s have a balanced back and forth about this. + 6 dB or whatever. Cheers. |
@amir_asr , thank you for sharing your perspectives. I have an education in computer engineering. But being an audiophile, I just don’t agree with the position that the science and measurements can totally explain our perceptions. I have a fundamental belief that science cannot explain all of the dimensions that impact our subjective interpretation of physical sound waves. Why? You have suggested folks get upset when ASR rejects a component that they subjectively praise. I think you are correct in many, many cases. It’s why people get so fired up about ASR. In my case I don’t care if ASR rejects a component that I subjectively enjoy - that doesn’t bother me in the slightest because if I enjoy it that’s all that matters to me. So why do I follow my subjective judgement over science? I simply don’t believe that science can FULLY explain, at our present level of understanding, how sound waves are subjectively interpreted by humans. Why do I believe this? My own blind testing experiments, over decades, that offer results that are not explainable. Differences in power cords, the effect of burn-in, vibration control products etc. Sure there are theories why these things make a difference even when blind testing, even with generalized theories that are proven, but to my knowledge none are proven in the specific application of HiFi audio. My engineering buddies think I’m absolutely nuts when I say music servers and USB cables make a difference in audio, but, with their knowledge, I can’t blame them at all for coming to those conclusions, as I have the same knowledge. But the difference is, they’ve not had the blind subjective test experiences that I’ve had, which leads me to question whether science really knows everything about the manner in which humans subjectively perceive sound waves. Without going into details that may be taboo on this board, I don’t consider myself to be a particularly spiritual person, despite the objective evidence that our universe could be part of some grand design. But if I had some sort of personal subjective experience that clarified a particular spiritual path for me, I could see myself heading down that path. But nothing remotely close to this has happened. But in the world of audio, it has! That’s why I feel we need to go beyond the science! What we perceive through our senses is not at all a “fantasy world” as you have suggested. That “fantasy world” is actually all that we have. Sure, measurements can tell us what is happening in the world. But no measurement, as of yet, can explain how we perceive it.
|
@milpai , I am only advocating that people behave maturely and stick to discussing audio, free from personal insults, lies and libel. I am not sure what aspect of human hearing we do not understand you are referring to? I did a quick perusal but it is hard to make a detailed search here or at least that is a skill I do not possess yet. For understanding audio, we know a lot. However, we don't know, beyond general statements, what any one person will prefer though there are theories about environmental exposure (what those around you like) and also genetic wiring. It am not sure that is relevant though. One statement that Amir made, was that a tube amplifier with a non-flat frequency with a particular speaker is a flawed design. I do not agree with that assessment for many reasons. First is that the resultant frequency response, in room, may be preferable, even more accurate. Second is that at an individual level, preference trumps accuracy, so the only person who can state flawed, is the final user. Last is that there are many flawed humans in the chain of musical production that all recordings are inherently flawed. We don't know what the best final playback function is, though Amir could make the argument, that on average, for that case at least, flat is best. |
