The Audio Science Review (ASR) approach to reviewing wines.

ASR Review:
Over the years, I've read a small handful of ASR speaker reviews.  I've noted the following characteristics:

1. The primary focus of all the reviews were the measurement results
2. In spite of that, the essential imperfection of speakers means that the interpretation of these results is subjective.
3. Any listening impressions were brief and at the end of the review.  
4. All were based on hooking up 1 speaker only. I believe he only listens on-axis.
5. He may (perhaps always) apply EQ to the input signal and A/B the results.
6. The room (and likely, the equipment) used in testing is at his convenience.  For example, large speakers are tested in a sub-optimal room, rather than his listening room.
7. In the few reviews I've read, the listening tests were used to confirm the measurement results.
8.  I've not made an effort to see how non "domes & cones" speakers fare in his testing.

Conclusions:
It does appear that Amir is aware of and somewhat sensitive to the concerns regarding his review methodology.  If you value stereo imaging, his listening tests will probably be singularly unpersuasive.  I haven't (and probably won't) read enough of his speaker reviews to know if his measurement & listening results ever differ.

The testing methodology ASR utilizes is efficient, which enables them to review equipment much more quickly.  However, the knowledge gained is also limited by the process.  The results are thus most useful for the least discriminating.

@prof 

Sorry, not calling you out on this, but you said "ASR evaluates equipment based on objective criteria that has been found to predict certain aspects of sound quality".

I was hoping that you could point to a link/post/discussion on ASR covering how the reader would correlate measurement anomalies directly with SQ issues?  This is 1 of the 2 big holes I find in reading ASR reviews, so hopefully it's covered somewhere on the site, in depth.

For example, in a Stereophile-style review, the listener might say something like "trumpet was a little spitty in its high register".  In the measurements section, that might be correlated with a slight rise in the on- or off-axis response.  In ASR, the measurements go first, so you get the same data.  In the listening section, though, you might see that Amir tried EQ'ing out the slight rise and liked or didn't care for the result.  No indication of how the listener would perceive the original issue.

The other big hole is imaging, which is ignored in the monaural listening test.  This has 2 parts - the first, alluded to above - due to the loss of the stereo image with only 1 speaker in play.  

There's also the speaker design - box vs planars vs horns vs OB, etc.  Each of these have significantly different radiating patterns.  Their optimum measurement results should also be radically different.

How is any of this information being conveyed to the reader?

@prof 

To quote from my previous post - "the speaker design - box vs planars vs horns vs OB".

You'll have no problem in any gathering of audiophiles finding people who prefer Magnepans or Klipschorns or Linkwitz over box speakers.  These speakers and many others differ in basic method of sound production, on- and off-axis frequency response, radiation patterns, impedance curves...  All of them have their fans.  They all differ quite noticeably in the way they sound.  Any set of measurements that would match well with one of these designs would likely produce poor results with the others.  Based on your argument, knowing Amir's specific measurement criteria should tell us what kind of speaker he prefers (subjectively).  You can't simply average all the speakers together into a Frankenspeaker.  So, has he ever divulged his reasoning for the measurements he makes and how he established their usefulness?

@chenry 

Thanks for your response, but I'm aware that he's using Spinorama.  My question isn't HOW he produces the pretty pictures, it's how he decides what is important.

Forgive me if I'm incorrect in assuming your response is out of ignorance, but all of the speakers you listed are boxes.  I'd say traditional, but Klipsch (horns, founded 1946), Magnepan (planars, original design 1969), and MartinLogan (electrostats, founded 1979) are all US companies with very high profiles in the audiophile community for 46-79 years, long enough to considered traditional in their own right.

There are also newer designs, many using the sort of drivers you would recognize, that are configured to produce dipolar or bipolar radiation patterns.  There are even omnidirectional speakers on the market now.  All of these designs have advantages & disadvantages, proponents & detractors, and will measure very different "on paper".  It may well be that Amir avoids all of these issues by sticking to domes & cones in a box, though.

I did find a thread on ASR entitled "Dipole vs Box speakers" - 5 pages long, nothing from Amir, some skeptics, some converts.  I'm not suggesting any one design concept is superior, but if this is all new to you, you might want to explore listening "out of the box".  It could change your life (or at least, sound system)!