How much can be measured -- and how much cannot?


There has been a lot of discussion over the years on Audiogon regarding the measurement of components and other audio products. Some people claim everything is either measurable now or will one day become measurable with more sophisticated measuring equipment. But others say there are things in high end audio that will never me measurable and that measurements are really not that important.

Here is a typical example -- a quote taken from the Stereophile forum regarding their review of the Playback Designs MPS-5:

"JA 2/17/10 Review Measurements of Playback Designs MPS-5
Posted: April 13, 2011 - 8:42am

John Atkinson's 2/17/10 review measurements of the Playback Designs MPS-5 revealed less than stellar technical performance even though Michael Fremer really liked the player. I've included JA's closing measurement remarks below followed by the manufacturer's comments.

To my knowledge there was never any followup in Stereophile regarding the manufacturers reply the MPS-5 could not be adequately measured with traditional measurement techniques.

I believe Stereophile should respond to this reply in the interests of its own measurements credibility.

Len"

How important do you think measurements are? Are the ears really the only true arbiter?
sabai
03-19-12: Ahendler
Actually I have never found measurements to help me in any audio decision to by a certain product. Measurements are helpful in the design process but do not tell you anything how a component will sound.
They won't tell you that the component will sound good, but they will often enable you to predict that the component will sound bad in a particular application, as a result of the kinds of mismatches that have been cited.

Basically, they allow you to narrow the field of potential candidates for purchase, to minimize trial and error, and to reduce the likelihood of expensive mistakes.

Regards,
-- Al
To me the best speakers give me an emotional connection to the music. The final of the Sibelius 2nd will give me goose bumps on certain speakers. I know of no measurement that will tell me which speaker will do that for me. Remember early solid state and early digital cd players measured great but sounded terrible. The statement above that measurements alone will tell you if you will like a certain speaker is just nonsense
"Despite audiophile claims to the contrary the science of of sound reproduction is extremely well understood"

No doubt. One is always at risk when proven scientific principles are ignored or not applied in solving a problem.

Here are the most useful audio specifications/measurements for me that I tend to always take into consideration:

Input and output impedance so these can be matched between components for better dynamics and lower distortion.

In addition, for amps, I look at power ratings, damping factors and current delivery related specs to get an idea of how well an amp might be able to drive a particular set of speakers.

Sensitivity related information can be useful as well,especially when dealing with various phono section designs and speakers.

I do not usually pay too much attention to distortion specs. I will look at various charted lab measurements if availble from a reliable source in order to gain more insight when possible.

That's mostly it that I can think of off hand.

I would say that a basic understanding how to interpret and apply audio metrics is a very advantageous skill for any true audiophile who wants a systematic means of improving their sound reliably to develop. It's something I am still working on and I try not to lose any sleep over these things though I probably have in practice from time to time.
Regarding the example in the introduction regarding John Atkinson's measurements of the Playback designs MPS-5, I would have been interested in hearing a discussion between the reviewer and John Atkinson as they listened to the PD together after the measurements were taken and after John Atkinson wrote about his observations.

Another recent example comes to mind -- John Atkinson's recent measurements of the AMR DP-777 opposite the reviewer's observations. John Atkinson expressed disappointment with the measurements of the AMR DP-777 where the reviewer praised the unit very highly.

Is it possible for the twain to meet? Would it not be interesting for John Atkinson to sit down and listen to some music with the reviewer when all is said and done to see if he finds the actually listening experience as irksome as his measurements? I think it would be interesting to hear their cross-talk.
This is like Rashomon. It is amazing how many differing points of view there are on this subject. Each one seems to be a valid part of this complex puzzle. IMO. The question remains, which aspect is the predominating factor -- or is this a matter of co-important factors?
"Audiophiles focus on what sounds good"
Of course we do if it does`nt sound good why bother?
I don`t know what your priorities are but I only want audio components that allow me to 'enjoy' the beauty and emotion of music. If you can`t connect to the music what are you listening for?
Regards,
I think the original question and all the responses miss the central point of the modern audiophile pursuit. Despite audiophile claims to the contrary the science of of sound reproduction is extremely well understood, particularly if we're talking analog signals. Audiophiles focus on what sounds good, which has become more a function of marketing and mass psychology. As a group we have been indoctrinated and conditioned to think about sound quality in very specific patterns and a group of talented entrepreneurs are profiting by exploiting these patterns. The products we buy are luxury goods and marketing involved is well understood - product differentiation and market segmentation predominate.

Audiophiles have two fundamental challenges to their world view, musicians and engineers. Each group basically thinks we're delusional. To paraphrase comrade Marx, "who you gonna' believe, me or your lying ears?"
Post removed 
Emotions and logic are two different contexts. Perhaps both are needed to fully evaluate and appreciate equipment and systems. For music, the final evaluation is how much is emotionally moving to the listener. One can ask, "do I hear detail, staging, extension, depth and evenness of frequency response," but in the end, whether any of these characteristics is emotionally involving is the ultimate question and challenge, and for that, to my knowledge, no quantitative measurement has been, so far, predictive.
>But others say there are things in high end audio that will never me measurable and that measurements are really not that important.

Speaker preference has a very strong correlation (if you weren't a scientist you'd say it was causal) with with uniformity of amplitude response curves at angles representative of a direct listening window and where front-wall, side-wall, and floor/ceiling reflections come from in average rooms.

Sean Olive has reportedly taken that to the next level producing a formula that predicts peoples' speaker ratings based on such measurements.

Stored energy is important too, although that shows up in polar response curves (ripples in the on-axis response can come from diffraction effects that aren't that audible, but ripples at many angles tend to be resonances which are) with fine enough frequency resolution.

>How important do you think measurements are?

Very. Good enough to predict that you're not going to like a speaker before you go to the trouble of listening.

>Are the ears really the only true arbiter?

If you can hear it you can measure it and if you can measure it you can fix it or decide it's not worth the price tag which goes with the fix.

You'd do well to read _Sound Reproduction: Loudspeakers and Rooms_ by Floyd Toole.
And I will add that learning to read and understand measurements will provide a great education to an audiophile. When I compare the specs I looked at originally compared to the factors that I try to take into account now, I realize how much I have learned from this hobby.

Off the top of my head, factors that I now take into account:
output voltage
output impedance
input impedance
input sensitivity
gain
harmonics distortion (2nd, 3rd order, etc)
damping factor
speaker distortion (hard to find in general)
crossover point
crossover slope (1st, 2nd, 4th order etc.)

I'd also add to that list the standard plots vs. frequency graphs that one sees in Stereophile's measurements section.

And when I see many of Al and others' posts, I realize that there is much more to learn.

BUT all of that having been said, I too ultimately come down on the side of letting your ears make the final decision. Psychoacoustics are pretty wild, can be tough to get your head around, and aspects of them are difficult/impossible to measure. Heck there are very likely aspects to psychoacoustics that remain to be identified. But trust your ears, they are excellent psychoacoustic instruments connected to all the bits of the brain that matter for a satisfying experience of music.
"The number of times that I and many others have referred to John Atkinson's measurements in helping Audiogon members to make purchase decisions or to diagnose problems is practically countless."
Actually I have never found measurements to help me in any audio decision to by a certain product. Measurements are helpful in the design process but do not tell you anything how a component will sound. Use your ears and they will guide you on the right path
Measurements are a beginning but they are not the end of the journey. It gives an idea of what the piece will do or can do then you have to figure out how that works with what you have and or are willing to acquire to make it work for you.
I second Mapman's comments. As I see it the main usefulness of measurements, and also specifications, is that they allow one to identify and RULE OUT candidates for purchase that would be poor matches to either the rest of the system (e.g., impedance incompatibilities, gain and sensitivity mismatches, etc.) or to the user's requirements (e.g., maximum volume capability, deep bass extension, etc.).

Also, they can facilitate diagnosis of issues that may exist with components that have already been purchased, but that may not be optimally matched to the rest of the system.

The number of times that I and many others have referred to John Atkinson's measurements in helping Audiogon members to make purchase decisions or to diagnose problems is practically countless.

A secondary benefit is that measurements and specifications can help to provide a better understanding of the design concepts and approaches that underlie a product, which is something that I always prefer to have.

Regards,
-- Al
"How important do you think measurements are?"

Important, but do not tell the whole story. Not likely that they ever will in the foreseeable future.

"Are the ears really the only true arbiter?"

In the end, for the end user, yes. No two ears along with the rest of the human auditory sensing system behind them are exactly the same, just like stereo gear. What else could possibly take their place?

Specs and measurements are most useful to help determine what gear might/should play well together up front during the decision making process, but its all you and your ears from there.