Good read: why comparing specifications is pointless


 

“ … Bitrates, sampling rates, bit sizes, wattages, amplifier classes…. as an audio enthusiast, there are countless specifications to compare. But it is – virtually – all meaningless. Why? Because the specifications that matter are not reported ánd because every manufacturer measures differently. let’s explain that...”

 

 

128x128akg_ca

Agree comparing specifications is completely pointless. The article pretends to answer the question when really it is an infomercial that winds up saying our specifications are okay, it's those other guys specs that are pointless.

Truth is all specifications amount in the end to pointless distractions, for the simple reason nobody knows which specifications matter, and by how much, compared to all the other specifications. 

Post removed 

Measurements matter in all aspects of life - audio is not an exception! When I heat a pizza in an oven I don't stick my hand in and feel how hot it is! Ovens have thermometers for that!

Post removed 

No but you sure stuck your tongue on a frozen flag pole.

Reading is fundamental. 

Guess ASR is your bible. 

Why? Because the specifications that matter are not reported ánd because every manufacturer measures differently. let’s explain that...”

When I heat a pizza in an oven I don’t stick my hand in and feel how hot it is!

I think Alpha Audio inadvertently made the opposite case. You need to have independent third-party testing to make sure the product you're buying at least matches manufacturer's published specifications. If the manufacturer provides none, then it should be tested and measured to ensure it's at least designed properly and functions as expected.

 

The measurements performed at ASR, Erin's Audio Corner, Stereophile, and SoundStage Hi-Fi (to name a few) typically use a consistent method for obtaining those numbers. In that case, you can compare the performance using those measurements because you know the measurement parameters.

 

Post removed 

He is saying simple "spec numbers" are no good and he is right.  You want to see full measurement graphs so that you can see the tested parameter varied.  Of course this is what I do at ASR day in and day out.  Here is the power measurements of an amp:

 

Note how I provide references of a mediocre and stated of the art amp at the same time to give you context.  Without it, you can get lost in see of numbers he is showing you.  

He then makes a bunch of other comments about power mattering at micro-watts which is neither here, nor there.  First, he calls them "distortion" when in reality that part of the graph (sloping down) is completely dominated by noise.  Distortion only takes over when the graph starts to move up which only happens near clipping (exception are amps without feedback or distortion-factory ones).

I also don't believe in this "first watt" business.  Speakers have become smaller to fit in homes better over the years and this has made them much less sensitive.  As a result, 1 watt is not going to do anything for you.  For this reason, I have standardized on 5 watts as a low value power output:

 

Notice the comprehensive dashboard that not only gives you distortion, but also the amplifier gain and full spectrum of noise+distortion.  The latter lets you apply perceptual analysis to determine audibility.  

Bottom line, yes, run fast from any single value specs thrown out there without detailed data as I have presented.  As a practical matter, for non-speaker products, you do need to stick to one source of measurements.  In my case, I have tested over 1000 audio products of this class in the last few years.  So you can compare with confidence.

For speakers (and headphones to some extent) we have quality measurement standard in the form of ANSI/CEA-2034.  With this data, you can indeed compare one source to another. This is what it looks like:

 

In on graph you get on-axis and (important) off-axis response.  And directivity.

You don't get distortion so I have added that with two (and sometimes three) standardized levels:

 

I have over 200 speakers measurements (I think) and others have posted measurements using similar scheme.  That hole then is being filled nicely although the process remains more expensive than for electronics.

Specs, in order of importance IMO:

1) in room FR

2) in room FR

3) in room FR

Unless you get that one the way you want I would agree, the rest of the specs are pointless.

You have to know what you are doing with in-room frequency response.  Otherwise, you have more than useless information.  Dr. Toole says it best in his wonderful book: Sound Reproduction: The Acoustics and Psychoacoustics of Loudspeakers and Rooms

 

Mind you, you absolutely need to measure your room and correct (the right) frequency response errors.  Just don't chase measurements without fully understanding how our perception works and difference between one microphone and two ears.  The internet is replete with wrong measurements and approaches to rooms.

Unless you get that one the way you want I would agree, the rest of the specs are pointless.

Who are you agreeing with, yourself?  Only in audio some folks think knowing less about what they are buying is a benefit to them.  Get the manufacturers to back their fidelity claims.  

That aside, OP said nothing about specs not being useful.  Measurements tell you if devices do nothing for your sound.  How much noise an amp can generate.  How much power it has, etc.  If you don't know this then you have a lot of learning to do.  

I am agreeing with "Good read: Why comparing specifications is pointless"

You take Revel or some other good measuring speaker, put it in a room with hard flat surfaces and you don’t need a FR chart to know that it ain’t singing like it could.

What is the point of having a golf swing panther DAC in a headless panther room? You tell me...

 

 

@amir_asr

The measurements you do on speakers is actually the least useful, see:

Acoustic measurements for our uses are of three principle varieties.

  1. • To measure the acoustic response of a loudspeaker, usually exclusive of the environment. (this is ASR)
  2. • To measure the acoustic sound field in a room, usually inclusive of the response of the loudspeaker. (this is my point in the above post)
  3. • To measure the noise level in a space.

The first two are closely related measures since room response and loudspeaker response are tightly coupled. The first measure is by far the most common, but, as we shall see, they are usually inadequate as an indicator of how the second measure will come out. Of course, what we want is a correct room response

(Earl Geddes: Premium Home Theater Design and Construction, CH 5)

@kota1 

I am agreeing with "Good read: Why comparing specifications is pointless"

That was the clickbait headline.  You didn't bother reading or understanding the article saying and showing measurements are absolutely useful. It is the difference between useless "specs" and real measurements.

You asked me:

Who are you agreeing with, yourself?

and I answered in my post that followed.

Now you answer my question:

What is the point of having a golf swing panther DAC in a headless panther room?

@amir_asr

The measurements you do on speakers is actually the least useful, see:

Nope.  They are incredibly useful in predicting listener preference of speakers in room. That is why those measurements are standardized.  And considered critical for speaker design.

As I noted, you absolutely need to measure your room as well.  It is just that the knowledge of what on there requires extreme care.  See for example this article I published on room reflections you talk about: 

You take Revel or some other good measuring speaker, put it in a room with hard flat surfaces and you don’t need a FR chart to know that it ain’t singing like it could.

What is the point of having a golf swing panther DAC in a headless panther room? You tell me...

I did tell you.  You need to spend time actually learning what sound reproduction in room involves.  Measurements of rooms can generate incredibly bad decisions especially when it comes to "treatments."  Ask any top acousticians what they think of DIY efforts and they will universally say they are done poorly.  And that people slap this and that on the wall, routinely creating dead rooms.  First thing they do is rip everything out and start over.

Audiophiles can learn to treat room modes however.  There, measurements do tell the truth.  Fixing those when combined with an excellent speaker creates wonderful results.  No need to go and create an ugly room full of padded walls and such.

@amir_asr 

OK, so please share the measurements of YOUR room so we can all benefit from how you address sound reproduction.

You state "audiophiles" can learn to treat room modes. Does that include you???

As for looks try this:

 

@amir_asr 

OK, so please share the measurements of YOUR room so we can all benefit from how you address sound reproduction.

This is not a thread about my room.  Or any room for that matter.  It is about audio gear specs/measurements.  You seem to have no ability to stay on topic.  

But maybe you share Earl's room measurements since you quoted him.  Do you have that?

You state "audiophiles" can learn to treat room modes. Does that include you???

I did my learning many years back.  I now I teach others. Suggest picking up a copy of Dr. Toole's book if you really want to learn the topic. Otherwise, please don't waste our time googling/cutting and pasting stuff.  It is too obvious that you lack any experience in this field.

 

Yes it is pointless unless one knows how to apply the numbers properly.  

@amir_asr , I walked the walk, my "nodes" are fine as you can see in my profile. As for yours, IDK what you are teaching but apparently you still have a lot to learn.

I suggest you pick up a copy of Geddes book "Premium Home Theater Design and Construction", you don’t even have a decent room for your gear yet from that pic @prof shared.

As for Toole, come join us in the FR thread, I have some stuff posted that may help you get your FR straightened out, that graph @prof posted of your room looked a little sloppy, you are abusing the DSP, it isn't designed for a total makeover.

 

 

@kota1 

@amir_asr , I walked the walk, my "nodes" are fine as you can see in my profile. As for yours, IDK what you are teaching but apparently you still have a lot to learn.

In your profile?  You mean this?

 

If so, you have so much to learn.....

@amir_asr

I have no idea what you gain from your deceptive site and posts. I like charts, I was a practicing scientist for over a decade. I have been a professional engineer and led large global technical groups and teams that evaluate and implement real world systems that run large corporations, high tech companies that produce some of the most advanced technology on the planet. But simplistic charting like you do completely miss the point of high end audio. You are not capturing the essence of the real problem but side tracking those that actually want to learn about this complex and interesting pursuit. You are adding no value for folks that are attempting to build great sounding systems.

Still waiting @amir_asr , if I got a lot to learn you say you are a teacher, get busy and post the goods, otherwise just more hot air, yada, yada, yada

Audiogon Forum 1- ASR 0

 

@ghdprentice , he doesn’t have the goods, he is deceptive because he has to be, if he had the goods he would post them. He wrote a full page boasting about his creds and not even one mention of his own gear?

If he had even a halfway decent system he would be just as braggadocious.

Since he can’t post what he doesn’t have he just points fingers to distract attention away from his own system.

I did offer him a rematch though, we’ll see if he actually has any game (but I suspect not)

I think he is just jealous of my "nodes" :)

 

@ghdprentice 

I have no idea what you gain from your deceptive site and posts. I like charts, but they must explain things that are not easily explained otherwise. But you are adding no value for folks that are attempting to build great sounding systems. 

Measurements absolutely show things that are not clearly "explained" otherwise.  Take a speaker I measured recently:

You should be able to easily tell the troubles in the 2 to 3 kHz even if I had not noted them.  Things like directivity may be foreign to you but it is all explained in every review.  I hope you agree that know if a speaker has colorations/faults is very important to know.  And not something you are going to get out of the random Joe reviewer.

If you don't understand these measurements, then you can watch videos I have done where I explain it all.  Here is one:

 

If you have specific concerns, let me know and I will address them. Until then, huge number of audiophiles have benefited from objective and reliable information on ASR.  We have grown to double the size of this site despite being much younger.  That didn't happen because people share your opinion.  It happened because they learned the information, modified their purchasing behavior, and are much happier for it.

@kota1 

I think he is just jealous of my "nodes" :)

You have no idea of your "nodes."  You are not understanding what Audyssey Pro software is showing you. It is not remotely what you think it is.  You are completely in the dark right now with respect to your room response post EQ.  Even pre-EQ, you don't know what you are looking at.

@amir_asr ,

sore loser

Audiogon Forum 1- ASR-0

If you want to win the game you gotta be in the game, post already.

 

Sounds just like those idiots in DC. 

This is not a thread about my room.  Or any room for that matter.  It is about audio gear specs/measurements

  

@kota1 

@ghdprentice , he doesn’t have the goods, he is deceptive because he has to be, if he had the goods he would post them. He wrote a full page boasting about his creds and not even one mention of his own gear?

My system and link to it including measurements was has been posted multiple times.  Here it is again:

 

If he had even a halfway decent system he would be just as braggadocious.

No, I am a private person and don't want to walk around bragging about my own system.  I only post it out of necessity where a test requires it as it did here: 

I let you go on thinking I don't want to post it to see how far you go with it.  Boy did you go far even though so often the answer was given!  You truly lack common sense and skills to have debates like this.

@kota1

I think he is just jealous of my "nodes" :)

Once again, you have no idea of how your room measures. You post this:

 

This is the output of Audyssey Pro. You made no measurements of your own so have no idea what this represents so let me explain.

On the left in black are what Audyssey measured as your room response. This shows the usual room modes and other response errors. Alas, there are no scales. You have no ideal how big or small those variations are. Further, you don’t know how much smoothing is applied which again, can hide issues.

Audyssey proceeds to create its correction filters. Those filters are based on what is shown in red on the right. It however, NEVER shows you the response after applying those targets!!! That’s right. Your room response is not remotely like that. Those red curves are the "targets" used, not what is achieved.

Don’t believe me? Here is a test of Auddyssey I performed with my system/room where I show what it says it did versus reality:

 

Black is the response with no EQ. Light pink is what Audyssey claims to have done. The reality/measured response is in red. As you see, they are NOT at all the same. And that is with me applying a lot of smoothing to actual measurements to help make it look similar.

Most importantly here, you have screwed up and did not create the proper target curve. Audyssey by default makes two major mistakes:

1. Uses a flat target curve. This is perceptually wrong and will make your system very bass light post correction. The proper target curve is what I show in my measurement above. It has increased bass response to make it sound more balanced and natural. Formal research shows that without this fix, Audyssey actually makes the system sound worse, not better:

 

This is the result of listening tests from that study:

 

Audyssey using the same target curve you did is #6 all the way to the right. It produced lower listener preference than no EQ!!!

2. You left the so called "BBC dip" in the target. This is that little notch in mid-frequencies. It is a mistaken belief that speakers in general are too hot in that region and so response should be pulled down. This is completely wrong. You need to override it and make that flat like the target I created (light pink).

Net, net, your system is improperly equalized. And you do NOT have any kind of measurements for you to understand it. Learn the topic for heaven’s sake.

@amir_asr , congratulations on posting something, it took a while but you stepped up. I look forward to comparing notes.

If you want to share your thoughts about my system, great. I’ll do same about yours later.

As for Audyssey I don’t think you are accurate about the software. You are using the app from the phone, that isn’t what I use at all. My software is PC based. As for Audyssey Pro there are different versions of the software and maybe you used one of the older versions. The version I am using was just updated this year. The chart I printed is before (black) and actual after (red), red is not some random target curve.
(I knew you were jealous of my "nodes").

You are thinking of ARC where the red is indeed just a target. Look at the graphs on my front height speakers, that red after curve wouldn’t be a desired target, that is what it is, the actual FR.

Now, why did you blow that big wad of cash on a golf swing panther speaker and stick them in that headless panther room with hard flat surfaces? Why not save money if you knew you couldn’t treat the room and just get one of the less expensive models?

As for the amps are they broken as a member here mentioned? They look great anyway.

Unless you step up your room I think you will never get a truly excellent result, just serviceable.

Next question, why are your speakers at that particular angle from the MLP? I set my front L-R speakers at the dolby specs of within 22 to 30 degrees. I moved the MLP forward in the room and have the same distance from the speakers in front of the room as the ones in back.

As for goals as you see in my overview I am primarily a MCH listener for music and movies, 2 channel only about 10% of the time in that system.

How did you position your MLP and why? It looks too far back from the pic.

 

 

 

 

@amir_asr , one more thing, if that is the setup you use to review MCH processors and receivers you need to stop reviewing them. You should review gear after using it as it was designed to be used, as well as measure it. If you want to review MCH gear use a MCH setup.

You spent $100K to measure a speaker in a way no one will ever use it. Now spend $1000 on the room that you use your speakers in every day. (I recommend Auralex, YMMV, they have specs and measurements on their website for all their products)

Audiogon Forum 1- ASR 1

(you get a point for showing up and sharing)

I have no idea what you gain from your deceptive site and posts. I like charts, I was a practicing scientist for over a decade ... You are not capturing the essence of the real problem but side tracking those that actually want to learn about this complex and interesting pursuit. You are adding no value for folks that are attempting to build great sounding systems.

This guy @amir_as is just the latest Youtuber wannabe guru to use Audiogon as a tool to drive traffic to their website where they hope to generate revenue. It’s no more complicated than that and there’s no reason to take them seriously. 

@amir_asr indeed, all the stuff you posted about Audyssey is for the consumer version with the $20 paid app. The version I use has a different license (around $200), a calibrated microphone with a serial number that gets linked to that license, and software for PC. The chart in my profile is generated so an installer can show a customer the before and after (actual). The db and hz legends aren’t in that chart. They are in the PC software when you customize the curves before I save and upload them to my processor though. You must have thought I was using the $20 app or some other type of software.

For the Denon receiver you posted about you can get a license for the upgrade for that unit at the Microsoft store.

As for your comments about Audyssey it is a shame that you haven’t gotten to try the update yet for your Denon receiver. See:

and

 

It is a good read, but also an advertisement for there own measurement 'skills'. The problem is a lack of standards in specs and measurement methods between companies and reviewers. Comparing specs can be usefull. You don't combine for example a SET amp with an Apogee speaker. Specs, like wattage or sensitivity matter, to name a few. So I don't agree with the author that is pointless to compare.

The Alpha Audio guys are just starting with measuring, and are trying to learn something. That is always a good thing in audio. It is journey for all of us.

 

@cleeds 

That must be it. To me, it just looks like fun with charts. I used to use charts to understand the performance of an organization. I’d find correlations then dig down to the cause and find it had nothing to do with the organization, but some the accounting system’s allocation mechanism. I found all sorts of correlations that turned out meaningless. In chemical and radioactive abundances in a previous career where they were interesting but useless as a predictor. 
 

My systems are shown under my UserID

@cleeds @ghdprentice @kota1 you all nailed it and are calling a spade a ♠️.

I have never been on the ASR site but have seen the musings of Amir’s followers, and that told me enough. Watched on YouTube video of Amir’s and that was enough for me.

 

@kota1 

@amir_asr indeed, all the stuff you posted about Audyssey is for the consumer version with the $20 paid app. The version I use has a different license (around $200), a calibrated microphone with a serial number that gets linked to that license, and software for PC. 

I have had the Pro version for some 15 years.  Everything I told you applies to it.  You must use the editor to create a proper target curve.  And learn to measure yourself to understand what it is really doing. 

The fact that you post that output thinking your room has that kind of flat response shows that you have no idea what you are doing.  All you did was run some automated calibration and blindly convinced yourself it must be perfect now.  Instead of posting here, learn to measure your room. If you don't know how, I have written a tutorial on Room EQ Wizard:

Until then, you have NOT post a single measurement of your room response.  None.  Come back when you have one.

@jerryg123 

@cleeds @ghdprentice @kota1 you all nailed it and are calling a spade a ♠️.

I have never been on the ASR site but have seen the musings of Amir’s followers, and that told me enough. Watched on YouTube video of Amir’s and that was enough for me.

How about you go and read a review on ASR and watch a video or two before forming an opinion?  They will be devoid of this kind of bickering I am responding to. 

 

The software was updated in 2022, it is a new version. Get the upgrade for your Denon, good luck.

@tantejuut 

 The problem is a lack of standards in specs and measurement methods between companies and reviewers.

This is part of the problem. The biggest issue is that companies are either not measuring, or have rudimentary/obsolete gear for doing so.  Perfect example is Schiit.  To save money they had bought this old, obsolete audio analyzer.  Once they saw my measurements, at first started to complain, throw rocks, etc.  Then they got their senses and not only bought the same Analyzer I have, but substantially improved the performance of their gear.  They now publish Audio Precision reports for every product they release -- quite admirable.

Even major companies like Denon are using and producing same measurements as me although sadly they are not releasing them to consumers.

A big barrier is the cost of this gear.  THe hardware is around $30,000.  For a company though, it is reasonable to purchase  but they don't feel pressure yet.  But they will as the value of measurements continues to broaden to more and more consumers.  

Oh I have and that is because they get kicked!

How about you listen to the equipment you review instead of just measuring befor formulating an opinion. 

How about you go and read a review on ASR and watch a video or two before forming an opinion?  They will be devoid of this kind of bickering I am responding to. 

@amir_asr you said:

Even major companies like Denon are using and producing same measurements as me although sadly they are not releasing them to consumers.

OK, let’s take a look at the Sound United "Experience Center" where they test the gear as it is meant to be used, in a proper setup. You have two speakers in an untreated room with a mic and a PC. Your in room FR that looks like the Mississippi river during a hurricane.

You don’t have the proper conditions to even listen to MCH gear much less review it, you know that, please stop, ok?

 

I went to the ASR site about six months or so ago. I read many review (most of the content) and conclusions. I particularly focused on components I had personal experience with. From a detailed analysis of method, reviews and conclusions I realized it was perhaps well intended but naive, misguided and misleading. To the beginner, it embodies the essence of what it takes to send them in the wrong direction if they wish to get good sound quality for a budget.