Amir and Blind Testing

I believe that when Amir places an audio device on his test bench, he goes into "lab tech" mode, where sound only exists to confirm the perfection of his measurements.

Seriously, improvement is a hard thing to quantify.  If everybody agrees A > B, life is easy.  But what if you're the only one who thinks so?

In theory, this is a great advantage for the objectivist.  Instead of days of pain-staking listening, they spend a few hours running a battery of tests.  But, wait - how do you evaluate the results of each test?  How do the tests correlate to the SQ characteristics most important to you?

Some people need that kind of assurance, so ASR definitely serves a purpose.  However, if you're trying to decide if you'll like the SQ of a specific component in your particular setup playing your kind of music, I doubt that ASR will be much help.

@noske FWIW, I have a great deal of respect for scientists - I double majored in physics and mathematics.  I have no problem with the tests, but I'm glad you like the pictures!

Rigorous testing is always important in technology, where the goal is the production of identical, interchangeable units.  However, we're so far from "perfect sound" that we're still arguing over which imperfections matter and how much.  The first measurement we targeted was flat frequency response.  Nope, we're not there yet.  So what's worse - a broad suck-out of the midrange, or an extremely high-Q resonance above the range of hearing?  What if the HF causes your amp to oscillate?  What if it's at middle C instead?  Is THD better than IM?  What about TIM?

The answers to these kinds of questions depend on the observer and the other components in the system.  Just curious (I've only looked at a couple ASR reviews), but does he have different acceptability criteria for tube vs solid state?  If he measures speakers, what about planars vs. box speakers?  It's pretty easy to set up your testing standard to fail either group pretty much entirely.

As I said, ASR serves a purpose.  If it works for you, that's great!  Clearly, it's not everyone's cup of tea, though.

I prefer reading Stereophile (I'm guessing that Absolute Sound made this transition also when they hired Robert Harley).  They generally do a subjective test and comparison to reference & similar-priced gear, followed by bench testing that includes listening in an attempt to corroborate both the subjective & objective results.  They always let you know when there are differing opinions on a component.  The bad news is that the industry has gotten too big to cover in that depth, which has resulted in perhaps an undue focus on the most expensive gear.