Good read: why comparing specifications is pointless


 

“ … Bitrates, sampling rates, bit sizes, wattages, amplifier classes…. as an audio enthusiast, there are countless specifications to compare. But it is – virtually – all meaningless. Why? Because the specifications that matter are not reported ánd because every manufacturer measures differently. let’s explain that...”

 

 

128x128akg_ca

@ghdprentice 

 I would not begin to consider evaluating a single new component without listening to it for a couple months.

I appreciate that you think you need that much time to evaluate a component.  But you and I are not similarly situated for many reasons:

1. I understand the full design and architecture of what I am testing.  This allows me to focus on what their weakness and strengths are.  An example of the former is a powered speaker.  These routinely have amplifiers that run out of gas before their drivers do.  So I test for that.  I am not just shooting in the dark thinking any and all things need to be evaluated.

2. I use measurements which help immensely with #1 above.  They show me objectively and reliability where I need to look.  If a speaker has a dip in 2 kHz, I use equalization to fill that.  I then perform AB testing to determine how audible that is.  Measurements are quick.  Electronics/tweaks take an afternoon.  Speakers take about a day.  With that in hand, and knowledge of the product, I am able to make very rapid progress in listening tests.

3. I have tested well over 1000 devices in the last 3 to 4 years.  That has enabled me to build methods and systems for fast and reliable comparisons.  For example, I have special music tracks that instantly tell me how well a speaker reproduces sub-bass.  I know how the competitors to the speakers perform relative to what I am testing. 

Audiophiles and "professional reviewers" throw random music at equipment with no aim or direction.  So no wonder it takes them so much longer to learn something about the product.  At the end they may just be guessing.  

4. I am professionally trained critical listener.  I also know psychoacoustics and research in this area that says long-term testing is completely unreliable.  See the digest of this AES paper:

Here is the punchline there:

The results were that the Long Island group [Audiophile/Take Home Group] was unable to identify the distortion in either of their tests. SMWTMS's listeners also failed the "take home" test scoring 11 correct out of 18 which fails to be significant at the 5% confidence level. However, using the A/B/X test, the SMWTMS not only proved audibility of the distortion within 45 minutes, but they went on to correctly identify a lower amount. The A/B/X test was proven to be more sensitive than long-term listening for this task.

Or if you are more comfortable with video, a complete tutorial in listener training, my ability find small impairments and explanation of above paper:

 

5. Adaptation.  Our brain adapts to its environment.  Think of the your computer fan running.  After a bit, you forget about it.  This is adaptation in play.  Same thing happens with say, a speaker that is bright.  Listen to it for a while and you adapt and no longer think it is bright.  It becomes the "new normal."  This is why speakers rank the same in formal studies regardless of the room they are tested in. Your brain learns to listen through the room.  From point of view of reviewing, you want to give the true nature of the sound, not what you have adapted to.

Dr. Toole explains this effect very well in his wonderful book:

I could go on but I hope you get the message that I follow the science and research in what I do.  What you and other reviewers do is based on lay impressions and what others have told you.  You have no proof point that you are creating reliable results.  Indeed, research shows as I post earlier, that professional reviewers are terribly unreliable in their assessment of speaker sound.

So you do what you want to do.  But unless you can prove your methodology to be right, and better, there is no argument here. 

@jerrybj 

Why would you even bother coming here Amir?

Well, it certainly isn't for hugs and kisses from the few of you this way!  😁

I came here because someone here created a thread saying Audio Science Review is a "cult" and is not open to any contrarian views.  Since then, he and a few others have been working overtime to prove this forum is that way!  There has also been a lot of misinformation which as you see, I have been correcting. 

This thread is about measurements which is core to what we do at ASR. Perhaps for that reason a poster flagged me yet again so here I am. 

Why are you in this thread?  If you are not valuing measurements, and don't want to read my posts, surely you would want to hang around rest of the forum than this one.

Go back to where people appreciate what you say, and reinforce what you believe to be true. Hopefully a few others here follow you.

 

I am pretty sure people appreciate what proper audio science and engineering says about audio in here as well.  A number of people have chimed in public and in private.

But yes, I do wonder at times whether my time is best spent elsewhere.  To the extent there are no more misinformation is posted about me and ASR forum, I may indeed fade away....