I Am Tired of Bogus Measurements

@kota1 these guys are talking about measurements to level match before you A/B speaker cables of the same Gauge, which doesn't make sense if there is no difference in cables as they claim,  how would it be louder with one cable than another if as they say no difference than no level match should be needed.

So are after market power cords, vibration control devices and power conditioners bogus if they measure well and make the system sound better? Does it win a prize instead? 

https://nordost.com/downloads/NewApproachesToAudioMeasurement.pdf

That paper is brutal.

1x CD player, older, $4000 ---  I have an idea, how about tell us the model. Why is that a secret?   It was obviously a unit with troubles that conveniently measured poorly. I am making a leap there? You bet, but I have a reason. That reason?  A current, $250 Japanese CD player was better than their $4000 unit, quite a bit from the pictures shown. They needed to pick something really awful to show a difference. They gamed the test. Worse, they gamed it in another way I will mention later.

29 graphs, count them, 29, not one of them with an adequate scale being used, let alone always with a good description and most of them, with absolutely no scale at all.

They have some pretty big claims of timing errors, 20-40useconds, though how someone could replicate this with almost no data I have no idea. Of course, they are using what could be an ancient (and purposely chosen) CD player, and to make it as bad as possible, they are using a CD made on a computer CD writer, not even a pressed CD. At this point now very old CD player, well worn, and a computer made CD, with "sample sized", their words, timing errors. It sounds like the CD was misreading. I could see a mechanical platform improving that. Personally not played a CD in over a decade. Rip everything to storage.

That paper is brutal.

All papers are brutal, I cited that one as an example. I did an in home audition using their method, it worked! Go figure.

Not all papers are brutal. Some are good, some are average, some are questionable, and some are really brutal. Both gaming the results with a really awful, but hidden source device, and a ton of graphs without scales, no pictures of the setup, limited details on the processing other than trust us, etc.  It is particularly bad.

@thespeakerdude

You argue too much. If you want to at least appear credible post your system and in room FR measurements.

@kota1 "I Am Tired of Bogus Measurements"

-how can be audio devices designed, improved, manufactured, and serviced/fixed  at the end without measurements? 

@westcoastaudiophile

how can be audio devices designed, improved, manufactured, and serviced/fixed at the end without measurements?

Bogus is when someone claims the sound quality of an audio device can be reliably predicted by measurements. When that happens it becomes a race to the bottom of who can replicate a measurement at the lowest possible price and claim superiority=bogus claims. The bogus reviewers and websites are promoting measurement hysteria as a form of click bait.

Is their a universal measurement for sound quality?

Look at all the dacs this guy bought and then sold and he explains at the end of the day why he feels the measurements weren’t the whole story:

@kota1 +1 on "Bogus is when someone claims the sound quality of an audio device can be reliably predicted by measurements”.

measurements are focused on single device compliance and performance, not complete sound system + room integrity. 

@westcoastaudiophile

measurements are focused on single device compliance and performance, not complete sound system + room integrity.

I find that when my room integrity measures well the sound system is subjectively better to my ears. The thing is you can’t predict this based on how a speaker was measured or an amp was measured. That's why I find people promoting measurements as an avenue to SQ as bogus. I find it has to be curated for each room relying on acoustics, not an anechoic chamber.

Even BEFORE EQ my room integrity is pretty good:

Can you imagine how bogus it would be for me to say  that whomever uses MY speakers will find they are a perfect match for their room and gear and offered my measurements above as "proof". That would be B-O-G-U-S. That is similar to what the measurement hysteria crowd is pushing as a S-C-A-M.

@kota1 , if you want to be appear credible stop posting links to reports / white papers / etc. that have either been ripped apart, or very questionable in content. I would also not posts a corrected room response and indicate it is the real corrected room response when it is a calculated room response.

I would also suggest not deflecting and answering the questions posed, not the obvious deflection that has nothing to do with the content.

Your repeated bating that has nothing to do with the topic also lacks maturity, but I am sure you know that.

@thespeakerdude 

You argue too much. If you want to at least appear credible post your system and in room FR measurements.

@thespeakerdude

This is a very well respected company known for unbiased measurements and product reviews, I am sure you can find a high quality, good measuring speaker quickly HERE, you’re welcome! 

Audioholics was bogus in trying to cancel an entire brand in Synergistic Research, here is the lead designers response:

ASR tried cancelling GR Research using bogus measurements so another reviewer showed up to check it out personally. What happened? What do you think:

Golden Sound used bogus measurements to try and cancel MQA, then another reviewer responded who wasn't even affilated with MQA. What happened was completely predictable, it sounded "very convincing" to him, bogus measurements and all:

All of these bogus websites attempt to use click bait and label themselves as the "doorman" guarding the high quality club. When put under scrutiny they just move on to their next target using bogus measurements as "proof". In reality they are simply using measurements to promote their bogus opinions.

@thespeakerdude   🤣🤣🤣

That is 4 strikes @kota1 ,  Hans Beekhuyzen is not an expert, I suspect he is not an expert on anything. That video does nothing to negate the work published by Golden Sound (which I came across when researching MQA a while back). If this is the best rebuttal to a very extensive test, that indicates to me that Golden Sound is probably right. You don't negate technical arguments with words, you negate them with technical arguments.

As soon as Hans starts to call into question something as fundamental as Nyquist, you know he is highly unqualified to make any statement about the topic though he tries to recover by stating no one has proved him wrong.  Not content to make himself look bad trying to call into question, Nyquist, he then starts talking about distortion and filters, claiming all filters make distortion. Marginally true, but also linear distortion, which we are much less sensitive too. Good thing or every speaker every made would sound horrible. Try comparing the linear distortion of a speaker to a DAC. Oh boy, that would be an eye opener for Hans!  He then makes a statement that we should be using 192Khz, though no one seems to be even able to prove conclusively we need more than 44.1, but perhaps he is not aware of even the most basic things about DACs and ADCs that most of us working near the technology can pick up by osmosis. Maybe I am wrong, but I suspect Hans is not very technical.

But all that aside, he makes the claim that most people prefer MQA to not using MQA. Funny story, but researching that is how I became more aware of Audiogon. I searched the web, and the most frequent comparison was Qobuz and Tidal. It was not overwhelming in Qobuz's favor, but significantly more preferred Qobuz in my non scientific internet review.

ASR tried cancelling GR Research using bogus measurements so another reviewer showed up to check it out personally. What happened? What do you think:

I think Strike 5!

For one, that was probably the worse wine/audio analogy ever made. I always picked my wines purely on the alcohol content, color and grape type, and not the 25 other major chemical compounds that contribute to taste.

So we are on to two. Two two two .... I guarantee those two crossovers will not measure the same. Just by looking at it, I know the iron core inductor will be much lower resistance than the air core inductor that replaced it. Also, those enormous film capacitors will have different ESR at frequency also impacting the final response. Perhaps at a gross level they will be similar, but not in detail. The changes will be as substantial probably more than your worry about component tolerances.

How else do I know this is not done by what I would consider a truly experienced professional? Start with the flat wire inductor. They look so high tech don’t they? I would almost want to use one just because they look good. Their reason for existence is a claim of reduced skin effect, even up to 100KHz. First, why do I care about the woofer part of the circuit at 100KHz. They claim less self heating due to space factor and winding density, but each wire layer is heavily insulated and the total surface area is less than a standard wire wound inductor meaning that in practice, a standard air core is superior thermally. I could go on about highly variable winding diameter, etc. but I think I said enough. I will finish with the itty Miflex capacitor next to the enormous Miflex capacitor. They are both +/-5%. What exactly is that small one supposed to do? It looks like it would be about 1/100th the value. Is that supposed to be a high frequency bypass? You will not find that on the crossovers of high end speakers for a reason.

Interesting sidebar:  Do you know why audio film capacitors are always 250, 400, 630V even though audio signals are never anywhere near those levels?

@kota1. Now you have me really confused. Are you now saying that measurements are good as long as they validate your subjective experience?

@yoyoyaya

All of the measurements should be considered bogus that are promoting an "angle" or self serving point of view. Obviously a manufacturers measurements can be self serving to promote their product. A reviewers measurements can be used to promote traffic to their site to generate revenue. Both players may have an angle, I call bogus on both.

I auditioned, I liked. Screw the "measurements. I found it interesting that Nordost published an article that confirmed my experience. I had that setup long before Nordost even acquired QRT from the founder. That’s all. I would not recommend buying something because of a measurement, only auditioning.

Interesting how they used a "chain" of devices to generate the results. I didn’t believe them because of their measurements, I believed them because of my in home experience.

Another reason measurements are bogus is reviewers pass judgement without actually using the product. In this example the measurements are kept in context of actual usage:

The AV8805 is a significant engineering achievement, both in the complexity of the design (look at that back panel!) and in its measured performance.

This next reviewer never even used the product as part of a home theater as intended:

Conclusions
Performance here is not awful but clearly could be a lot better as sister group Denon has shown. $5,000 is a ton of money for an AV product so performance needs to be much more optimized than it is. .

Bogus, anyone buying equipment can skip both sets of measurements. One guy likes it, the other guy "meh". You need to audition at home.

@thespeakerdude

I always picked my wines purely on the alcohol content,

That explains it, no wonder why

You argue too much. If you want to at least appear credible post your system and in room FR measurements.

🍷🥤🥃🍻🤤

This device is NOT bogus and is known to give very accurate measurements:

The thing is you can’t predict this based on how a speaker was measured or an amp was measured

Much more accurately, YOU cannot predict much about a speaker and how most people will feel about the sound when the room is known, and a full range of measurements are available. A lack of experience does not extend to all people.

However, even for other products your statement is still YOU, not everyone. Many amp vendors, D'Agostino, Pass, Macintosh even claim they tune their amps specifically for their sound (some have a few sounds) and that is based on a transfer function that is unique to their products or a group of their products. Given there is no way at all they can test their amplifiers with more than a small subset of speakers, but achieve reported consistent results, they have a good handle on the measured performance.  However, just a few topics over, you are advocating 100% for matched amplifiers to drivers for active speakers, which is 100% a measurement exercise. That position and your statement I quoted are at odds with each other.

@kota1. Now you have me really confused. Are you now saying that measurements are good as long as they validate your subjective experience?

I would reach that conclusion.

Conclusions
Performance here is not awful but clearly could be a lot better as sister group Denon has shown. $5,000 is a ton of money for an AV product so performance needs to be much more optimized than it is. .

Bogus, anyone buying equipment can skip both sets of measurements. One guy likes it, the other guy "meh". You need to audition at home.

The conclusion appears to be based on expecting a $5,000 AVR to have internal DAC performance maybe somewhere in the ballpark of a $100 DAC. That does not seem unreasonable, or are you saying that all DACs sound the same as long as they are half decent? If you play a lot with external digital volume control, it may be important to you to have DAC performance in your $5,000 AVR that is better than a $10 phone dongle but maybe it does not matter if they all sound the same. With that level of engineering detail, what else have they missed?

If you need that device @kota1, maybe I should go a little easier on you, then again, making jokes about alcoholism, a serious disease for many, and genetically influenced is in pretty bad taste on your part.

@thespeakerdude

Did you take your measurements yet? 

Dr. Sean Olive on why good measurements are so difficult:

What is the most challenging thing about what you do?

"The biggest challenge is designing listening tests that can provide accurate and reliable ratings of sound quality."

Who is doing these measurements and where are they reliably tracking the data? This seems like it might actually be useful.

Excellent post @kota1, it completely makes the point I stated above. Here are some key statements in it.

In your most recent paper, you proposed a statistical model that predicts listeners’ preference ratings of headphones. How did you first come upon the concept of the model?

We now understand what the target response should be for achieving good sound.

The statistical model for predicting listeners’ preference ratings of headphones based on deviations in its frequency response was really an extension of a similar model I developed in 2004 for predicting listeners’ loudspeaker ratings. The only difference is the headphone ratings are based on a single curve whereas the loudspeaker’s radiation uses several curves to characterize its sound over a sphere.

"Damn.....ran out of popcorn...." 🤨

@kota1. You still have me confused. Are you saying that:

(a) measurements are bogus i.e. false and untrue or

(b) that some people use accurate measurements but make bogus claims on foot of them or

(c) that some people use bogus i.e. false measurements to support bogus claims?

For clarity please state which of the above (or combination) you are arguing.

Hold on.  What are BOGUS measurements?  What do you mean by the term BOGUS?

Dictionary definition is NOT GENUINE, COUNTERFEIT, SPURIOUS.

Is that what you intend?

Obviously no-one wants to be supplied with measurements that are FALSE.  But you mean measurements that in your opinion are not useful.  That is an entirely different kettle of fish.  FALSE means objectively false.  Stuff you don't find useful is just your opinion.

@clearthinker

From the OP:

The cancelling of brands, the talking down to the customers, is bogus.

Just posting measurements or specs in the context of a component is fine.

There is no universal measurement for sound quality so using measurements to predict sound quality is COMPLETELY bogus.

@yoyoyaya

None of the above.

Some people accurately measure things that are not relevant in the context of the sound quality of a product and has 0 predictive (or bogus) value to a consumer. Then they try and misconstrue the result as being predictive. Ranking products based on SNR that is beyond human hearing is an example of this That is bogus.

Some people will cherry pick measurements to come to a conclusion that was predetermined. Certain websites promote themselves as being the arbiter of what is and what is not snake oil. If a product they deemed useless measures exemplary they will either bury the results or slam the lab that did the testing. Bogus. All of the videos I posted of reviewers being called out by other reviewers or the companies they tried to cancel are examples of this. Measurements that don’t align with the outcome they want are demonized or ignored. They are being bogus with no regard except one, drive traffic to their website.

@OP

Sorry I'm not sure what 'cancelling' of brands is.  Is it a Generation Z term for disagreeing or ignoring?

Talking down to customers is not bogus, i.e. false.  It is just rude and rather negative for sales penetration.

I agree measurements are not predictive of sound QUALITY overall.  But they can often be accurate predictors of aspects of sound CHARACTER, eg a substantial suck-out or a kink in the frequency trace of a speaker.

It is very rare that snake oil type products display any measurement changes in use.  If they did, their proponents would not have to say the difference needs to be heard in terrms of sound quality (a generalised non-scientific term even though it needs to be apprehended).

Dr. Floyd Toole stated "Two ears and a brain respond very differently to a complex sound field — and are much more analytical — than an omni-directional mic and analyzer."  A very significant point from a man that should know.

I spent many years chasing the best measuring audio products within my budget. Some eliminated via in store audition, some traded after at home performance fell short.  Listening always the final arbiter. 

Many years spent designing and testing complex electronic systems.  Measuring and testing was excruciatingly long and detailed process.  The last hurdle was performance in the intended environment.  Measurements provided a reasonable predictor of performance, but never 100%.  

@kota1

All of the videos I posted of reviewers being called out by other reviewers or the companies they tried to cancel are examples of this.

You batted 0 out of 5 on the videos you posted that I looked at. All were very flawed if not outright wrong. The videos you posted really highlight the need for competent technical sites to do the work they do as there are far too many people and companies publishing information that is misleading if not outright wrong.

@clearthinker 

Sorry I'm not sure what 'cancelling' of brands is.  Is it a Generation Z term for disagreeing or ignoring?

"Cancel culture is the act of canceling a brand, public figure, or company that you disagree with.  When a brand is canceled, support is withdrawn and consumers spend their money with competitors if the brand or company has said something offensive. The brand is canceled through online shaming on social media platforms"

It is very rare that snake oil type products display any measurement changes in use.

I don't see measurements for that many products except MPG for new cars. Generally specs are provided instead.

@texbychoice 

Many years spent designing and testing complex electronic systems.  Measuring and testing was excruciatingly long and detailed process.  The last hurdle was performance in the intended environment.  Measurements provided a reasonable predictor of performance, but never 100%.  

Much respect, good example of being legit. The bogus measurements that are used to promote a product or promote the person doing the measuring=bogus. 

Measurements are what they are, if the suit is a 38 and I'm a 38 will it fit. Will I like it. who knows. Will my spouse like it? LOL. 

You got to try the suit on, look in the mirror, and ask your wife.

You gotta bring gear home, audition it, and probably ask your wife (LOL on the WAF). There is NO measurement that can predict how a product will pass the hurdles.

@kota1 

Thank you for that definition.

But surely it should be 'something which you think is offensive'.

Taking offence is rather like judging sound quality.  We each do it differently and far too much importance is attached to what at best is entirely subjective.

@kota1

Repeatedly members on this site say that measurements don’t matter, and that sites like ASR have no influence in the audiophile community.

If that is the case, how could those sites possibly "cancel" a brand that caters to audiophiles.

I don’t think you have provided any good examples of a bogus measurement. The only one that comes close is in regards to a $5,000 AVR, were the performance was good enough, which was recognized on one graph, but the overall conclusion was that the high price was not justified given the high price. I have a hard to interpreting that as cancelling.

There is obviously the ongoing argument about what is audible, with the science sites taking a stance that they feel can be supported by science. I think it is a fair argument that in some cases system level issues such as system level noise, perhaps recognized, are overly glossed over. You could argue the conclusions are even too black and white for the average non technical reviewer. You are going to have far more success with these as arguments as opposed to something easily dismissed. If you care about the validity of measurements, why not address that with Amir as opposed to making assertions that are easily dismissed?

@clearthinker

It goes both ways, bogus measurements can be use to cancel a brand or to promote one as well. It depends on what you measure and the context of how the product is used. It doesn't have to be offensive to be bogus.

Summary of this lengthy topic. Bogus is whatever Kota 1 wants it to be.

On that basis, as they say over here on Dragon's Den, I'm out.

@yoyoyaya     Yes, Kota misuses the term BOGUS, despite my efforts to explain.

And now he has suggested I elided offensive and bogus, which I didn't.

I can tell you all one thing: there was never a fraction of the offensive and bogus stuff before they invented the internet.  We are fast approaching the stage where it does more damage than good.  There never was such a leveller down.  The few decent people that are left will soon be running a mile.

Ho hum.

@clearthinker 

there was never a fraction of the offensive and bogus stuff before they invented the internet

+1, now you know why I am tired of it. Click bait using bogus measurements to drive traffic to your site and SELL more stuff, yep.

Saying bogus measurements repeatedly does not make it factual Using bogus links and bogus white papers to assert other things are bogus also does not make it factual. Also not proven is any evidence of a cancellation. Criticism is not cancellation especially if a relatively consistent basis is used for all criticism even if you do not like what that basis is.

The "measure" of this thread is quickly approaching zero. 🤣