What should be mandatory in every professional published review-


When testing a company's newest amp, preamp, etc, and it is a refinement of a prior product that was on the market, ie, a Mark II, an SE version, a .2 etc, it should be mandatory that the review includes a direct comparison with the immediate predecessor. IMHO, it's not enough to know ion the product is good; it's also important to know if there is a meaningful difference with the immediate predecessor.

I'm  fan of Pass Labs, and I just looked at a review of an XP22 preamp. I find it very disturbing that there was no direct comparison between the XP22 and the XP20. And this lack of direct comparison is ubiquitous in hi-end published reviews, across all brands of gear tested. I don't blame the gear manufacturers, but rather the publications as I view this as an abdication of journalistic integrity.

 

Opinions welcome- 

128x128zavato

Note that reviewers live in a world of allegedly quantifiable differences as they have access to myriad items that the great unwashed do not. That said, and having said that, your and my ears hear different things and I for one have proven to myself the value of my tastes over all, as I tend to use my own earballs to make audio decisions.

@wolf_garcia Exactly.  And that’s why the product comparisons in reviews are so critical (and why I hate TAS reviews as they don’t bother to do them) as they give you relative context that at least to some degree enables you to calibrate your hearing versus the reviewer’s.  

There is no way to "calibrate" your hearing...you simply have to hear things yourself as (previously stated I thought) generally you not only haven't heard the item being reviewed at length in your system, you likely haven't heard any of whatever the other compared items are, again, at length in your system. What was that quote?, "Writing about music is like dancing about architecture." Maybe from Martin Mull...in any case, again, if the writer is interesting I feel lucky, but there is no proof unless I can get my hands on the pudding.

Interesting thread.
Those folks who have the “immediate predecessor” product might be able to glean certain insights from a new product review. That may come from a variety of sources, such as the reviewer’s personal biases in reproduced sound (and whether they resonate with those of the reader) as well as prior reviews. For example, I like a lot of what PHD of TAS reviewed because I have similar biases to his own. It’s just a flavor for what might be of interest to me if I were in the market for that particular item covered by the reviewer. I suspect reviewers develop a faithful following based on the contents of their reviews, regardless of product comparisons they might make. So if the OP demands certain requirements of the reviewers that he follows, then that’s his preference. But every reviewer has a different audience who might differ in what they want from a given product review and reviewer. Mandatory requirements? Maybe for certain audiences yes, but by no means not all audiences.

There is no way to "calibrate" your hearing

Yes, there absolutely is.  By triangulating the sound of components you’ve heard, other reviews of products used for comparison in the review, and the reviewer’s perception of other components you can glean some valuable relative information.  Of course it always comes down to your own ears, but by combining information from reviews along with other information you can at least get some indications of certain sonic characteristics of a component that can be helpful in a reader deciding what they might or might not want to consider or audition.  To dismiss a review because the reviewer may hear a bit differently than you or have different tastes or because you haven’t heard any of the equipment in the review is shortchanging what you can take away from a review with a little more thought and reading.  I’ve bought several pieces using reviews in this way and have never been disappointed and have found I got just what I was looking for and expected.  It’s a useful skill to have as good high-end dealers are disappearing and being able to hear equipment before buying is getting harder and harder.

Nope, there isn't. And trust me....I read and think about this stuff WAY too much. You simply won't ACTUALLY know the sound of a thing unless you give a component time in your system...If a lot of reviewers say the same thing about something you can make reasonable assumptions, but that isn't actual "calibration," it's just allowing logical influence...I bought a Pass XA-25 after countless positive reviews knowing full well it would compete with a treasured and amazing sounding tube amp I already owned, and it did...lucky me as I didn't have to send it back. I think speaker systems are likely the most risky, and I've proven that to myself too many times.