What should be mandatory in every professional published review-

I think most of us know what we can learn from specific reviewers and which we can ignore.  Often based on personal preferences, and reviewers system, such as some that test electronics with only speakers that essentially have little bass...they all have flaws and also things we can learn..,.reviews help me learn about gear, not tell me what to buy...

@wolf_garcia I would certainly be interested in the ones that you can sit on, especially if it puts out the right amount of heat, hell it might cure what ails me. Enjoy the music

+1 @jl35 

at this point it is safe to say that most reviews are flawed and not an undeniable reference for purchasing anything.  they are constrained from mentioning anything remotely detracting about the sound and in fact barely describe the sound character at all. they simply cannot bite the hand that feeds them

That was not my experience.  In 15 years of reviewing I was never constrained from saying anything negative.  Over that time I only wrote one negative review, not because I was constrained but because if a product reaches a level where it gets a review it’s likely gotten very good feedback or comes from an established company that knows what its doing and doesn’t produce bad-sounding products.  Point is, reviewers almost without exception get gear that sounds good, which is the main reason you don’t read many negative reviews.  Reviewers don’t want to review crap products and magazines/sites don’t want to review them either.  Plus, in this day and age it’s hard to even find something that just sounds bad.  That said, a reviewer should absolutely point out areas where a product might be a bit compromised, which is why I always found product comparisons to be the most interesting part of any review as it provides extremely useful context for the reader. 

most of the text in a review has nothing to do with how it sounds.  they are mostly fluff that includes company history, the new technology and why it should sound better, room and system setup, and maybe a little about how it sounds

This I absolutely agree with and has been a pet peeve of mine for a while.  Too many reviews contain 80%-90% background info, specs, etc. and only dedicate a couple paragraphs on how something sounds.  Plus, in this day and age when anyone can go to a company website and get a lot of this information I’d rather a review to just include a link for the component under review and refer the reader to that to get certain info unless there’s something notable about a product’s design, specs, etc. that warrants further explanation.  Anyway, that’s my take FWIW.

Note that reviewers live in a world of allegedly quantifiable differences as they have access to myriad items that the great unwashed do not. That said, and having said that, your and my ears hear different things and I for one have proven to myself the value of my tastes over all, as I tend to use my own earballs to make audio decisions. Example: New Klipsch Heresy IV...bought a pair and decided that although they are somewhat better built than the IIIs I already had (better speaker mounting screws and binding posts, ports lowering the bass frequencies by 10hz or so, AQ wiring), the IVs redesigned mid horn (simplified poly drivers replacing titanium full throated drivers) had a high-mid frequency bump that sort of yelled out. I couldn't stand it so I sold 'em and am happy with my lovely IIIs. I also sent back some well regarded Sonists, and a pair of pretty ZU Omens that were sort of awful. How anybody likes Dirty Weekend speakers is a mystery, but I don't really care, and hey...what do I know anyway? All well regarded with great reviews, but until you have them in your listening space you simply don't know. For me the reviews should be well written and entertaining, with expected grains of salt available in any readers head.

Note that reviewers live in a world of allegedly quantifiable differences as they have access to myriad items that the great unwashed do not. That said, and having said that, your and my ears hear different things and I for one have proven to myself the value of my tastes over all, as I tend to use my own earballs to make audio decisions.

@wolf_garcia Exactly.  And that’s why the product comparisons in reviews are so critical (and why I hate TAS reviews as they don’t bother to do them) as they give you relative context that at least to some degree enables you to calibrate your hearing versus the reviewer’s.  

There is no way to "calibrate" your hearing...you simply have to hear things yourself as (previously stated I thought) generally you not only haven't heard the item being reviewed at length in your system, you likely haven't heard any of whatever the other compared items are, again, at length in your system. What was that quote?, "Writing about music is like dancing about architecture." Maybe from Martin Mull...in any case, again, if the writer is interesting I feel lucky, but there is no proof unless I can get my hands on the pudding.

Interesting thread.
Those folks who have the “immediate predecessor” product might be able to glean certain insights from a new product review. That may come from a variety of sources, such as the reviewer’s personal biases in reproduced sound (and whether they resonate with those of the reader) as well as prior reviews. For example, I like a lot of what PHD of TAS reviewed because I have similar biases to his own. It’s just a flavor for what might be of interest to me if I were in the market for that particular item covered by the reviewer. I suspect reviewers develop a faithful following based on the contents of their reviews, regardless of product comparisons they might make. So if the OP demands certain requirements of the reviewers that he follows, then that’s his preference. But every reviewer has a different audience who might differ in what they want from a given product review and reviewer. Mandatory requirements? Maybe for certain audiences yes, but by no means not all audiences.

There is no way to "calibrate" your hearing

Yes, there absolutely is.  By triangulating the sound of components you’ve heard, other reviews of products used for comparison in the review, and the reviewer’s perception of other components you can glean some valuable relative information.  Of course it always comes down to your own ears, but by combining information from reviews along with other information you can at least get some indications of certain sonic characteristics of a component that can be helpful in a reader deciding what they might or might not want to consider or audition.  To dismiss a review because the reviewer may hear a bit differently than you or have different tastes or because you haven’t heard any of the equipment in the review is shortchanging what you can take away from a review with a little more thought and reading.  I’ve bought several pieces using reviews in this way and have never been disappointed and have found I got just what I was looking for and expected.  It’s a useful skill to have as good high-end dealers are disappearing and being able to hear equipment before buying is getting harder and harder.

Nope, there isn't. And trust me....I read and think about this stuff WAY too much. You simply won't ACTUALLY know the sound of a thing unless you give a component time in your system...If a lot of reviewers say the same thing about something you can make reasonable assumptions, but that isn't actual "calibration," it's just allowing logical influence...I bought a Pass XA-25 after countless positive reviews knowing full well it would compete with a treasured and amazing sounding tube amp I already owned, and it did...lucky me as I didn't have to send it back. I think speaker systems are likely the most risky, and I've proven that to myself too many times.

i would also like more reviews that test in typical compact amuuurican living rooms of typical americans with a mortgage, not auditorium-sized venues in manses.  

"Gear that sounds good"

That is so different amongst listeners that it would help to know what the sonic preferences of the reviewer actually are.

You simply won’t ACTUALLY know the sound of a thing unless you give a component time in your system...If a lot of reviewers say the same thing about something you can make reasonable assumptions

@wolf_garcia And that’s the whole point. I’m not saying it’ll be certain that this will necessarily coincide with your hearing, but it certainly gives you a possible indication of what you might expect to hear or want to audition. Nobody’s saying the reviewer should be the final arbiter of what you should think because that’s ALWAYS up to you and your ears, but if you read consistent impressions by reviewers you can certainly include that in your arsenal as to what you might or might not be interested in. To me, that’s as far as the benefit of reviews should go given all the variables involved, but I hope you can agree that used for the information they can potentially offer within their limits they can still be very useful.

El redundo los bloviato a mundo.

El whatevero. 

I am probably idealistic, but I would like to know a reviewer’s ties and obligation to any company whose products are they reviewing, along with the publication’s. "Pay to play", payola, freebies, etc have long plagued magazines that review gear, and it calls into question the independence and integrity of reviewers and their publications. For example, some photography and gun magazines and their writers have been notorious about taking freebies from manufacturers, or doing glowing reviews on products and—like a miracle (cough, cough)—a few pages later you see full-page ads bought by those same manufacturers. For that reason, Consumer Reports buys everything they test iirc, so they are not beholden to manufacturers. The magazine business has gotten very tough, and advertising is the lifeblood, but magazines who basically sell glowing reviews in exchange for ads are whores. I don’t know which audio reviewers and publications are the best and worst in this regard (and I am not asking here!), but I have been told who is by trustworthy by people whom I trust.

It’s a matter of who can one trust, and that is enormously important to me. YMMV. I am not talking about subjective opinions, I am talking about opinions which are purchased.

I won’t name names, but a few notable veteran people in the audio industry have named to me reviewers they trust to be straight-shooters, and also those who are renowned for taking gear (either outright or in the form of "longterm" loans, or buying gear at cost) from manufacturers. One longtime retailer literally guffawed when I mentioned a very prominent reviewer who is apparently egregious in accepting gear as a quid pro quo (either that or he is a millionaire and can afford his $1,000,000 reference system/gear, and it’s possible I guess). I have had personal communication with a reviewer who has probably a >$150K (or more) reference system, and who admitted that most of it is very long-term loans from manufacturers. Hell, I can’t even get most dealers to allow an in-home audition without buying the item in exchange for a return policy (in their defense, they may have only on demo model of an item on hand). For that reason, the return policies of Music Direct, Audio Advisor and some other online dealers are great, but they don’t always carry what I am interested in.

If a publication is afraid to publish critical reviews (that doesn’t mean bad, it means honest), then they should say so. They should also disclose any ties to manufacturers or ways in which they are beholden. I have some experience with this working for the LA Times for 20 years. We were not allowed to accept anything of value over $25 from anyone we were doing a story on, which basically limited it to a modest meal. We had reporters who were fired for violating that code of ethics.

@patrickdowns I totally understand your skepticism and concern, and I can only speak from my own 16 years of reviewing, but I honestly think the corruption you’re alluding to is the exception rather than the rule — or at least I sincerely hope it is. Anyway, I can speak from my time at Soundstage! that I was never offered any free product or was incentivized in any way what to write except just to write about what I really heard. As I’ve mentioned before, by the time a product reaches a level of public interest that it merits a review it’s already gotten high praise or is from a well-established manufacturer, so the odds of getting a piece for review that sounds outright bad in any way is between slim and none. THAT’S the main reason you don’t read many negative reviews. Period. I only wrote one negative review in 16 years, and it ain’t cause I wasn’t ready to write more, it’s just that the products I was offered to review were by and large very good or excellent products. That, and nobody wants to review crap, so crap tends to never rise to the level of even getting a review. I think this could be the case of a few bad apples can spoil the bunch. But, that said, I’d say use your own internal BS meter to judge whether you can trust a reviewer’s opinions or not. Me, as a reviewer, if a review doesn’t make explicit comparisons to another competitive product I dismiss it out of hand because there’s no “fact check” to keep the reviewer honest. Many’s the time I thought I had a good handle on a review product’s sound only to be completely humbled when inserting a competitive product to find my radar was off and had to largely reshape and adjust my impressions — and the review — accordingly. Our aural memory just isn’t that good, or at least mine isn’t. Soundstage!, to its credit, wouldn’t allow a reviewer to review a component unless they had a competitive product available with which to compare it, and I think that’s the way it should be done to produce a truly rigorous and meaningful review. And I’ll once again call out The Absolute Sound (whatever the hell that even is) for going out of their way to not only not provide any direct product comparisons, but also in many cases not even disclose the components in a reviewer’s reference system so the reader has absolutely no clue what basis for comparison, if any, that reviewer was basing their conclusions on. Absurd. Just absurd and outright cowardly IMHO.

Sorry to drone on here, but having been in the reviewing trenches for many years and being a consumer of audio equipment myself I’ve developed some pretty strong opinions as to what a good and genuinely informative, thorough, and ultimately helpful review should contain. Hope this helps at least provide a little perspective and maybe just a touch less skepticism toward reviewers as most reviewers I know make next to nothing and do it purely for the love of music and to provide honest and helpful information/opinions to other fellow audiophiles. Just my $0.02 FWIW.

soix

THANKS for your perspective as a reviewer. I didn’t mean for anyone to think I’m making a broad accusation about the integrity of high-end audio reviewers— I certainly am not. Just as I took camera reviews in consumer magazines like Popular Photography with a bucket of salt, maybe the mid-fi and electronics publications are less reliable (I’ll use that euphemism) than high end pubs like AbSound, Stereophile, your former publication, and others. I will name one name. I had a fellow who puts together really expensive systems for people (up to a million $$) and who has been in the industry for many years tell me that Robert Harley’s reputation for integrity is unimpeachable. I would assume that, as Editor of AbSound, he expects the same of his writers. I hope so.

If a reviewer has some sort of a relationship with a company whose product they have reviewed (eg, they were allowed to buy the review sample at a discount) I just want to know. Even at a discount, they are voting thumbs-up for the product if they choose to buy it with their hard-earned dollars. That’s a valid endorsement, though some might think it’s inappropriate to get that discount because it seems like pay to play. FWIW

As to the rest of the debate, many good points made. I could never be a reviewer because I am too much of a people pleaser, and if I liked a company and its people I would find it hard to be candid about a piece of gear where they missed the mark and say so. Tough love is necessary, as people rely on reviews to guide them in buying sometimes very expensive gear which is often difficult to audition due to geography. I live in the boonies and have to travel hours to the big city, and sometimes other big cities to audition. 

Cheers

If a reviewer has some sort of a relationship with a company whose product they have reviewed (eg, they were allowed to buy the review sample at a discount) I just want to know.

@patrickdowns I’ll save you the suspense — all reviewers who write for a publication can get dealer pricing for a product they review (typically 25% to 50% off depending on the product type and if they’re sold direct or through a dealer network). Most publications/sites pay reviewers next to nothing for the many, many hours of work it takes to write a review from unboxing to finish, so dealer pricing is a meaningful perk that incentivizes reviewers to sacrifice all those hours to do what they do. That said, reviewers get to hear lots of gear so if they buy the review sample, even though it’s at a discount, it’s only because they feel that component is truly special and is about the highest praise a reviewer can bestow on a product. I only bought a review sample if it was significantly better than what I had in my review system, and since most reviewers have at least pretty good systems/components it makes quite a statement if something gets purchased after a review is completed. Anyway…

Hi soix

Most publications/sites pay reviewers next to nothing for the many, many hours of work it takes to write a review from unboxing to finish

I take issue with these terms of work, but I suppose it’s the reality in magazine/publication economics. The problem, as I see it, is it incentivizes reviewers to review gear they are personally interested in and anticipate buying. If the publication assigns the items to be reviewed, that makes it more neutral. I will admit that I would write a review of the Treo CT if I could buy them for a 50% discount! 😎

I only bought a review sample if it was significantlybetter than what I had in my review system, and since most reviewers have at least pretty good systems/components it makes quite a statement if something gets purchased after a review is completed.

As I said, it is an endorsement of a bought piece...the reviewer putting their money where their mouth is.

In a more perfect world, more magazines would be like Consumer Reports and would buy the gear they review, to maintain more objectivity. Prohibitively expensive of course.

I appreciate your candor and feedback (and you love Vandersteens too!).

I take issue with these terms of work

I encourage you to try writing a review for a publication and go through that whole process again and again before taking any issue.  I submit it’s not as easy as many people seem to think.  Not sure how anyone could not love Vandys, but then there’s no accounting for individual taste.  😝

@soix

I encourage you to try writing a review for a publication and go through that whole process again and again before taking any issue.

The reviewers at What-Hi? have got review writing down to a formula.

Nice large colorful photos and a few words that say absolutely nothing.

The words ’could be’ and ’might be’ tend to get used quite a lot.

It’s been the audio equivalent of Playboy magazine for decades.

Lots and lots of entrancing pleasures lie therein all glammed up to the eyeballs sumptuously lit. None of them will ever look as good once you’ve got them back home, but no worries, there’ll be plenty more different ones next month.

As a coffee table entertainment journal it’s perfectly acceptable.

As an audiophile review magazine it’s worse than useless.

Positively misleading.

The fact that it’s probably the UKs longest lasting audio magazine once again underlines the fact that readers prefer entertainment even when it’s masquerading as information.

I was surprised to find that it had been acquired by Future Publishing back in 2018. I guess the previous owners Haymarket felt the wind was blowing the wrong way and let the WHFs future survival become someone else’s problem.

That’s the trouble with these magazines, they’re not much use as information but we’ll still miss them once they’re gone.

Online magazines, at least for me, are even worse than online books.

@cd318 Ha! Agree 1000% re: What HiFi? and I’ll throw in HiFi+ as another useles rag. 

I have been in the Music  and Audio Industry for over 25 years and I myself would love to read some reviews written by member soix.

cheers !

@sts Just google Tim Shea and Soundstage a stuff should pop up. I reviewed for them from like 2002 to 2017. Ayre, Eggleston, Thiel, Bryston, Bel Canto, Acoustic Zen, Liberty (aka PBN) Audio, McCormack and a buncha other things. I recommend some caffeine if ur gonna actually read any of this stuff. Just sayin’.

Thanks -  This gives us a perspective on your taste and impressions.

soix,

You may have misunderstood me. The issue I have is with publications who don’t pay what I would consider a proper rate for reviews, and instead take advantage of the willingness of reviewers to do the work for a low pay scale with some promise of discounts on gear. If the writers agree to do so, it is their choice. I still think that if they are actually trying to earn a living they may be getting taken advantage of.

I’ll use an analogy, from my former profession—editorial photography. Publications (some/too many) offer assignments with low pay by telling the photog they’ll be getting great "exposure" and building their portfolios. That doesn’t pay the bills. I can’t eat photo credits. Sadly, many photographers starting out have taken the bait, therefore making it harder for established photogs to command/demand higher rates (even though they were paid it at one point). It’s a race to the bottom, with some photogs undercutting others and ultimately, themselves.

I don’t know how it all works in the audio reviewing business. In sports photography, there are more than a few weekend warriors, doctors or the like, who can afford $50,000 in camera gear and will give their photos away for free in exchange for the sideline photo credential. It’s done real harm to pros trying to make a living. Maybe there are rich trustafarians or well-heeled people doing the same in audio, enjoying getting a byline and a discount on gear. You might know.

Cheers

You may have misunderstood me. The issue I have is with publications who don’t pay what I would consider a proper rate for reviews, and instead take advantage of the willingness of reviewers to do the work for a low pay scale with some promise of discounts on gear. If the writers agree to do so, it is their choice.

@patrickdowns You are correct. Unless you write for the glossy mags you’re not gonna make money writing reviews. I did it because I love audio gear and writing, and the prospect of getting to listen to a lot of equipment in my room/system along with getting dealer pricing was something that was very attractive to me. But, after a while I got tired of humping equipment in and out of my system and spending tons of hours reviewing equipment to make someone else rich so I stopped but still very glad I did it. AFAIK the audio review industry doesn’t have any wealthy warriors undermining professional reviewers who do it as a career as in sports photography, but that totally sucks.

@johnread57 

where have I gone? Nowhere; just seeing how this has taken on a life of its own 

@soix

In 15 years of reviewing I was never constrained from saying anything negative. Over that time I only wrote one negative review

I guess everyone gets a prize, kinda thing. Everyone is above average. No normal distribution which may be described as a bell curve.

Never constrained from saying anything negative. No, a better way of saying it is that you were perhaps sufficiently rewarded by writing only positive things.

Fraud. Yes, you can quote me on that.

@noske So, you purport to know my actual experience better than I do?  You’re absolutely misguided and clueless, and you can quote me on that. 

@soix I know nothing of your actual experience.  I only know what you have written on these esteemed pages.

Should I be misguided, please share with the community here that in your 15 years of being a reviewer exactly why you provided only one negative review.

And for the sake of clarity and for the avoidance of doubt, define what constitutes a "negative review".

I know nothing of your actual experience.

@noske That’s correct, you don’t.  Yet you still write this presupposing my reviews were influenced to be positive…

Never constrained from saying anything negative. No, a better way of saying it is that you were perhaps sufficiently rewarded by writing only positive things.
 

As I’ve said before, at no time as a reviewer was I ever influenced to write anything in any way other than what I heard.  If I was I would no longer write for that publication.  Period.

Should I be misguided, please share with the community here that in your 15 years of being a reviewer exactly why you provided only one negative review.  And for the sake of clarity and for the avoidance of doubt, define what constitutes a "negative review".

Again, and as I’ve stated before, by the time a product rises to the point of getting a review it’s either a product from an established manufacturer who knows what they’re doing or a new product that is garnering a lot of interest due to good performance.  Either way, it’s very, very rare a reviewer receives a components that just sounds “bad.”  The one product I did write a negative review on didn’t sound good to me and I wrote it up as such, and had I gotten another product that disappointed I would’ve had no problem writing another negative review.  That said, no product is perfect, which is why I always fully disclosed areas where I thought a review component was better or worse relative to a competitive product.  Is that enough clarity for you?

@soix 

The one product I did write a negative review on didn’t sound good to me 

 Is that enough clarity for you?

Yes.

That the reviewer must listen to the product. 

@soix 

by the time a product rises to the point of getting a review it’s either a product from an established manufacturer who knows what they’re doing or a new product that is garnering a lot of interest due to good performance.

Well that just reduces the interest in any review to that of Bayesian logic and I copy from Wiki.  Dull, boring and handwaving reviews are just so not interesting.  Precise criticism and a skeptical approach is what I prefer.  But then I am not an audiophile.

Bayesian statistics is a theory in the field of statistics based on the Bayesian interpretation of probability where probability expresses a degree of belief in an event. The degree of belief may be based on prior knowledge about the event, such as the results of previous experiments, or on personal beliefs about the event.

Precise criticism and a skeptical approach is what I prefer.

@noske Just as I wouldn’t say something sounds good when it doesn’t, nor would I ever say something sounds bad when it doesn’t just to be “critical.” That’s the main reason why the product comparison sections are so critical as it’s possible to communicate what aspects of a review component’s sound was different, better, or worse compared to a similar product. To me, this is the most informative and useful part of any review and why I can’t stand TAS reviews because they rarely bother to do any product comparisons and I find their reviews to be “dull, boring, and hand waving” reviews I find next to useless. Again, just my take FWIW.

Certified results of a recent hearing test as performed by a certified audiologist.

A full-disclosure statement by the reviewer of said reviewer’s preferences in "how reproduced music SHOULD sound" as well as her/his taste IN genres of music.

And finally, a statement by the reviewer that addresses the nature of relativity and subjectivity.

@soix

Just as I wouldn’t say something sounds good when it doesn’t, nor would I ever say something sounds bad when it doesn’t just to be “critical.”

My bad.  I meant (in hindsight)  to convey the word critique,  Not criticism.

Must be a tough gig.

@millercarbon 

How long can it float? 

About a day until the funeral directors come in to collect.

After that it is the duty of the plumbers.

Room size, flooring (concrete, wooden...), windows (my room the front wal has a very large glass window, back wall is 3 sided floor to ceiling glass windows, walls: concrete, lath, sheet rock wood studs or metal studs...,  treatments, carpeting,window shades, curtains heavy duty or..., dedicated line or not, 

I think it should be mandatory that the reviewer disclose whether he gets the piece of equipment for free, or at a discount that he is reviewing

Pass Labs does not update models as quickly as some manufactures, so assume an upgrade is evident. There is no need to depend on reviews. Way too many self- serving variables to put any trust in a print or video reviewer. Think of reviews as a sales pitch to sell a new product. The previous model is on the second hand market which has very little financial relevance to the reviewer or manufacture. 

Bench test, for all specs against what says in the manual

current, and rms 8, 4, 2 ohm output