Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)


About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.

Yes, each meteor produced a sizzling sound, like the sound of a frying pan.

Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.

ASR would have agreed with this sound reasoning based in elementary science.

Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.

Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.

I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.

nyev

Showing 21 responses by cleeds

This issue isn't unique to ASR. We have members here who insist that what you hear doesn't matter, and that they can determine how a component will sound based solely on its photo or spec sheet. Things get ugly quickly if you question their "logic."

amir_asr

.... you want to live in a fantasy world where differences that don't exist or are below audible, are so obvious that your wife can hear from the kitchen ...

You live in a fantasy world. You don't know anything at all about me. But you have an active imagination, I'll give you that.

 

1extreme

They have so little regard at ASR for actually listening to components that they don’t even bother listening to them.

Exactly. The measurements and data have little value if not correlated with what we hear.

ASR does wield a lot of power and influence and can hurt a company producing good audio products.

I really don't think ASR has much power or influence at all. It's just a noisy group with grievances, which is very common today.

mastering92

Scientific papers are subject to peer review. Publishing the results instantly is not a great idea.

Quite so. Typically, a paper would undergo at least some informal peer review before it is even submitted for publication. When it is submitted, it’s then subject to a formal review before even being accepted for publication. When that part of the process is poorly done if often reflects poorly on the publication itself, so although mistakes happen, it’s exceptional.

There are those who wrap themselves in science, then use the pick-and-choose smorgasbord approach in practice. You see some do this with religion, too. Measurementalism is rather like religion in that it is taken on faith (there’s no need to listen!) rather than facts.

Of course other factors influence these YouTubers, too, as has been noted.

A lot of scientific work revealed to the public is not peer reviewed and all that is reviewed is method.

There’s a little more to it than that. For example, the logic and premise are examined, and that step alone would disqualify a lot of the "science" as seemingly practiced at ASR.

The peer review does not consist of other people replicating the work as you have suggested.

That is absolutely true.

Even with peer review, many poor papers are published and if you are presenting new concepts there may not be any one "better than you".

Yes, both extremes exist: papers that have little or no value, and papers that have enormous value.

What really happened to Pons & Fleischmann in the early ’80s (hint: the University of Utah really effed it up) explains what can happen when science goes wrong, even when respected researchers are involved.

 

prof

... It's like if you claimed that I need to come to your place to see your Perpetual Motion machine, and if I point out that current understanding of physics suggests your claim is so unlikely it's not worth my while to make such a trip.  You could try to dismiss my reasons ...

That's the logical fallacy of the Exluded Middle. Don't be silly.

If you want to point to specific examples to make your case, and you can show why Amir was unjustified in not listening...be my guest.

I've already done that. In my view, a test of an audio component is not complete without a listening evaluation. I understand you disagree, and that Amir disagrees vehemently. So be it.

Whether it’s Amir or me or some other like-minded folk, we are open to ANYTHING

If that were true, Amir would be sure to listen to every component that he "tests." That he sometimes manages to avoid listening - and has a whole bunch of wordy rationalizations to justify that - undermines whatever science he’s trying to pursue.

It is a mistake to presume skepticism or demands for good evidence equates to close-mindedness.

This is a hobbyist’s group, not a scientific forum, so your "demands for good evidence" really don’t belong here. If you don't like the evidence presented, it's really your problem. No one here owes you anything.

prof

Sure, that’s fine. Except that opinion offers no justification for your characterization of Amir’s position, re any specific instance where he didn’t listen.

The specific instance - something we debated at the time - is immaterial. My opinion, which I need not justify to your satisfaction, is that a review of an audio component that doesn’t include listening is of little to no value. And we all know that Amir doesn’t always bother to listen, and has extensive, wordy rationalizations to justify that. He has lots and lots of words.

What good does it if somebody tells others how a component sounds?

That’s for you to decide. Sometimes, as can happen with a movie or music critic, you find someone whose taste coincides with yours. That gives their opinion more weight. In audio for example, listeners may often independently conclude that a certain speaker tends to sound "bright," and that can be useful.

I need to hear it with my own ears and brain in order to make a judgement.

I agree completely, although I spell judgment differently. Of course, as has been shown in this thread, some dismiss those experiences completely. While that’s fine, it doesn’t obligate anyone here to submit to their demands for proof.

Someone’s high end acquisitions don’t give any more credibility in terms of making technical claims about audio.

That’s quite true.

... For instance, if you understood how the Klippel Analyzer System that Amir uses worked ...

Ahhhhh, but you make an exception for your favored shiny tool, for which I’m sure you’re willing to submit lengthy rationalization. Hey @prof, it’s clear you’re a smart guy. Do you not see your ill logic in this matter, or are you just funnin’ with us?

alexatpos

...I really, really cant understand what drives you, (or anybody else that shares your beleifs) in a attempt where you are trying to ’explain’ to people, who have different perspective or experience, that they are ’wrong’? I found that funny, but also very pretentious, none the less.

As @thyname also notes, those who behave that way typically are consumed with belief ("religion") and faith, by definition, requires no proof. The rigid, fundamentalist dogma leaves no room for the infidels who refuse conversion.

As with religion, some of the fundamentalist proselytizers are soft-spoken, engaging, and even appear thoughtful. Some are harsh, brazen, and confrontational.

Occasionally, one will slip up and reveal his agenda, as in this thread:

I have no personal agenda, other than to call out over priced, under performing products when I see them. Audio is full of over priced products that don’t measure up to far cheaper alternatives.

See, he has no personal agenda, he just wants to be the savior.

Does any consumer needs ’Klippel Anlayzer’ or blind test, or what ever else to determine what he actually hears?

Absolutely yes, according the measurementalist’s faith doctrine. That’s why some component evaluations don’t even include listening. After all, listening is such a flawed exercise.

We are adults.

Nonsense. It’s obvious that children post here and when their posts violate the rules, they should be deleted just as the adults are. If you claim to be the victim of attack, flag the post for the moderators. That’s why they’re here. Don’t use "victimization" as a basis for argument.

If you want to make this a faith and religion discussion and talk about proof one point of view will win every honest argument.

The fundamentalist measurementalists here have already made this into a matter of faith and religion, as others have noted, and you’re just piling on with a point of view (faith) that will "win" every time.

@prof is attempting to bring a balanced view and he still gets attacked.

I don’t see any attacks on @prof, although I’ve seen some from him that were deleted. In any event, attacks aren’t allowed under the rules here so there’s no need to play the victim card. Just flag the post for the moderators.

thespeakerdude

... Listening to lots of different systems does not give you experience ...

kota1

Yes it does, listening to lots of different systems does give you experience ...

Of course @kota1 is correct. What we experience very much matters. That is problematic for those who apparently want to define our experiences for us, as evidenced by their "arguments" in this thread that are increasingly subject to deletion.

What POSSIBLE evidence can you bring to someone like this, that they are wrong?

It seems to be your mission to convince others that they are wrong. That appears to be the root of the problem here and explains your increasing frustration.

This is the strong bias that so many bring to these discussions. They view their own opinions on how to evaluate gear as the default - e.g. "The Only Way To Truly Evaluate Gear Is By Listening To It, Like We Do" ...

Hey @prof that’s the logical fallacy known as the strawman argument. In fact, I’ve never seen anyone here make that claim, yet you put quotes around it.

In your next post, you opine:

...it’s clear that you have a bias operating that is causing you to place a very negative spin on someone else’s perspective ...

You commit the sin you blame on others. All your words can’t conceal that.

@prof is putting forth very valid arguments ... You may take issue with his response to other’s beliefs as is your right but can you fault his methodology and back that up?

I’ve quoted the ill logic he increasingly employs. There can be no better proof than that!

This is his latest use of ill logic in an attempt to conceal his previous use of ill logic:

You know very well the point I was making ...

Nope, it’s not my job to try and read his mind.

... Somehow you are too good for it whereas the rest of the industry is not ... Instead of trying so hard to stay confused, why not pay attention ... If you can’t reliably tell the cables apart, then don’t come and ask others to chase ferries. If you can, then you would have done something really useful ...

More ad hominem, appeal to authority, and bandwagon fallacies  from a measurementalist. As for the "chasing ferries" remark - you’re getting more and more colorful. Perhaps science is not your calling.

 

 

I follow science. You disdain the profession.

Oh no, not at all, you could not be more mistaken. Perhaps you have me confused with someone else.

I am all about what I can prove ...

If so, please be careful about whom you accuse of disdaining science.

Measurements are repeatable and provide valuable insight. Why else would you hate them? You must feel the mist of audio science brushing against your face.

What on earth are you talking about?

amir_asr

There is currently no research going on to validate what you all claim to hear.

Nonsense. There are many people striving to advance the audio arts. Some of them are right here on A'gon.

So do what you want. But please don’t talk to me about it.

No one needs your permission to participate here.