Does "Non Compensated" Reviewing Still Exist?


I still subscribe to two of the major "audiophile" publications. To be fair, I would rather not reveal the titles of these publications, so for the sake of reference let's just call one Stereophile and the other The Absolute Sound. I have subscribed to both (and other) publications, on and off, for over 20 years. I have also seen a few other publications bite the dust in the past 20 years. I don't really have any specific interest in the equipment reviews but they used to be a great reference source. Although, I find that the music reviews and new music release information is pretty thorough and diverse.

I just received the newest "XXXXXXXXXXX" publication including the "500 Great Sounding Audio Products". It's actually the "Recommended Components" issue. I decided to thumb through the articles containing said components and read the "final conclusions" to some of these reviews.

Below, I have provided some "conclusion" excerpts from the "reviewed" components of some past issues also.

This "review" excerpt was on a turntable:

"I'm not about to tell you to hock the mink and dump the Mercedes. It only makes sense if you listen extensively to analog LP and have a large vinyl collection or the patience and desire to build one—which today will take no little effort. If you do take the plunge, it will likely be your final investment (periodic cartridge replacements excepted) in analog front-end hardware. The sonic benefits are, in this reviewer's opinion, genuine. But to seek them out must be, inevitably, a carefully considered, individual decision."

Here's another "review" excerpt on a CD player:

"I've heard CD players that had better rhythm and pacing, more midrange liquidity and transparency, greater depth of field, and finer resolution. But I can't recall hearing a more musically involving, fulsomely detailed, three-dimensional presentation from any other CD player at such a modest price as the XXXXXXXXXX."

Or another excerpt on a preamp:

"The "XXXX" is a success after all, but a qualified one: It can play music brilliantly well, and it can be a very good value. Having spent more time with this pretty little thing than I usually do with a review sample, I feel unusually comfortable in recommending it—but now, all the more, I look forward to the day when the clever people at "XXXX" turn their attention to the comparatively cheap and electrically messy world that most music-lovers inhabit."

After reading these compelling conclusions, I have to wonder why I actually read the article in the first place. I also have to wonder what the hell they are actually saying with all of their wishy-washy, totally vague, substance lacking, non-committal crap.

Does real, non-biased, non-compensatory reviewing exist any longer? Is there anyone out there who still does a review in the manner in which these aforementioned publications USED to review? 20 years ago? When these guys didn't like something, they TOLD you they didn't like it, and generally substantiated their reasoning behind their opinions. Likewise, if they actually enjoyed a component. They would be more than willing to recommend a listen OR a purchase.

Now? They don't say anything! I find, by the time you get done reading these reviews, you have no more information from listening evaluation than when you started. I take ANY review with a grain of salt. Electronic components only sound "right" to that particular user, in a particular environment, with a particular synergy, with particular corresponding components. But, it would be nice to have some sort of FAIRLY accurate reference.

Here is MY conclusion to some of these conclusions:

"The "XXXXXXX" is one of the best sounding components of it's type. It will compete with any other component in the same price range, if you actually like the sound of the other components in this price range. When listening to classical music, the orchestra REALLY stood out. Rock music reproduced with this unit was VERY dynamic and loud. Jazz and Blues had exhibited a wonderful "Toe Tapping" quality.

If you are in the market for a component like this one, you really won't do better, unless you consider purchasing a better sounding unit for more money. It IS lacking the dynamics, frequency extension, quiet operation, and build quality of better sounding units, but other than that, it's right there with the best of them in it's class, if of course, you like it's class".

Is this an over generalization, or is this a fairly accurate evaluation? Do others feel the same? Are there review sources that actually COMMIT to their opinions devoid of their commitments to advertising revenues?

Has this become a thing of the past?

128x128buscis2
I've concluded audio is impossible to figure out. Better, worse, or just different, who knows? Heck, many times the differences amount to not much more than a feeling or perception. Getting real, defineable improvements require *large* increases in money spent, for the most part. Typically a 3X increase in cost by my estimation.
I appreciate being able to post less than positive comments on components and have never been deleted. So I don't know about "big brother".
I think most people here post exactly what they think with no financial gain in mind and even in such an "open forum" look at the varied perceptions of good vs. bad. No way a stereo mag can define what everybody will or won't like. Even if a reviewer says exactly what they think.
I'm not defending intentional deceit, just saying that you can't automatically say any component is good or bad.
There appears to be no such thing as right or wrong in audio. Only personal taste. Imagine how confusing regular life would be if we followed the same tao. Red lights mean stop .... unless you're late for work or it's your birthday. Speeding is illegal unless you're going to the beach. And on and on. Without rules and laws, society would fall apart. The audio world is one of anarchy it seems.
Unfortunately, however correct subjective reviewing is, I think it has thrown the audio world into the lunatic fringe (of which I am a member BTW) of consumer goods; Art Dudley giving a helping hand as of late, and resigned audio to an increasingly smaller sector of personal consumption.
Very good discourse in this thread!

I have laid back, as most of my views have been made clear in another interesting thread a couple of weeks ago.

Spencer you raise some very good points. I believe the answer for dealing with issues, in terms of shills lurking within us, is one having to prove themselves over time here.

Often, a person will come in all praise and bluster and he has never posted before. This has happened many times, the most easily remembered was the guy who began touting a modded $200 power amp, saying he had owned just about everything over the past 20 - 25 years, and this product was the best he has encountered. I jumped on him immediately, but he issued more posts where he stated his position, that he was not a shill, and that he apologized for offending anyone. However, within a few days, it turned out he was the distributor of the product, but somehow "forgot" to disclose that to all of us.

Yes, with our busy schedules these days, I can see how that happens all the time - "Yes, I own the speaker company, but somehow I forgot to mention it in all of the excitement..."

Again, for me, the thing is long term performance here. I think that like most things in life, you can only get to know a person over the long term. How they conduct themselves always is the final word in truth. It is pretty easy to see if someone is honest. Objectivity is not the most important thing, as this is a subjective field. But, fairness should be the rule.

As we have met in person, I think we both have confidence that each of us is passionate and true to the hobby. That is basically all I look for in a person, be they member, salesperson, distributor, or manufacturer. All are welcome and add to the diversity and experience of this site, in my opinion.
Electric Monk: I don't read Maximum PC but my business partner does. From what he tells me, it is a magazine that i would love... IF i was a "puter geek". It sounds like you have the same impression of that mag that he does i.e. head and shoulder's above the rest and able to do so while still retaining their integrity. Where are the audio rags like this??? Sean
>
I would love to have a current reference system and music listened to sidebar for every reviewer. It would make understanding their 'reviews' a lot more helpful.
As far as reviewing goes I wish the audio mags took the same tack as my two favorite computer mags. PC Gamer and Maximum PC (the same company produces both) are very informative in the review process. They have no problem rating games poorly (Terminator 3 just received a 23% rating) regardless of the company that puts it out.

They also have a column called the "Dog House" where users write in to complain about poor treatment by and shoddy products from manufacturers, either current of defunct. They clear up a lot of confusion for the readers by cutting to the chase and telling you what you want to know, not what the advertisers want you to be told.

One really good idea is a small bio column listing the current games and products that the reviewers are hooked on at the time giving you some insight into what type of games float a particular reviewer's boat (whether it be Sims, First Person Shooter, Strategy, etc.). Maybe we will get lucky and a mag like this will come along for the audiophile commmunity.
While I'm on top of my game here--

As time goes on the full effect that magazines have will be interesting to watch. The Y Generation, or whatever you want to call them, reads more magazines now than anybody else ever really did. They also, however, rely on the internet for more information than any other demographic. So where will this put the magazines in 5 or 10 years? I'm not sure if any of the magazines right now can adjust to these market changes-- especially the Absolute Sound, given the way they charge you to download past reviews. Heck, where will audio be in 5 or 10 years? These kids can be turned into audiophiles with the right entry-level products and marketing. But who's going to do it?
Sean--

The reviewers may have other jobs, but most publishers work exclusively as publishers. There are a lot of people working behind the scenes of a magazine that people don't often remember. THOSE are the people who are truly reliant on advertising dollars, not the reviewers.

Sorry I didn't make myself more clear. While it is true that a small reviewing team can put together a website or newsletter and review all the equipment that they want to, to publish a full-length 4-color magazine requires more funding than most people realize. And to keep that magazine successful (most fail in the first year) is even more difficult.
Sean...Good question. My reasons were as follows, generally in order of importance.

1. I am familiar with the technology (from work) and I know that it has great (even revolutionary) promise.

2. I researched the CarverProZR line of amps. This included getting the schematic off their web site. (How many home audio manufacturers show you their schematics?). I also got the technical info from Tripath on the digital controler, used by this and several other well-regarded digital amps.

3. The price was right. (I was tempted by Spectron, but the price was out of line). No other way I could feed those power-hungry Maggies, not to mention the multiple subwoofers.

4. Last, and least, favorable (sometimes over-the-top)reviews of the sonics. These did not make me buy the unit, but they "enabled" the purchase for the other reasons cited.

I have not been disapointed. As I mentioned in a prior posting, I screwed up some speaker cable wiring, such that one poor amp was working into a load of less than one ohm. The darned thing just kept pumping out the amps so that the sound from the speaker was "almost" OK, and it took me a while to figure out what was wrong. I don't recommend this wiring scheme, but it did show that this amp has balls.
El: if you don't think that "reviews" aka "opinions of others that are familiar with a specific component" are influential or give you an idea of what to expect from a product, what made you decide to purchase the Carver Pro amps that you are currently running? Sean
>
It's funny....when you get down to it, most everything is either driven by either sex or money!
Look at organized sports. I can remember seeing so many "fixed" boxing and fighting matches over the years, I think I might as well have watched WWE wrestling and been done with it!
But, back on course...you're right. As one who's worked in the audio business for many years, and is intimately familiar with many of the products that are ultimately reviewed, I've had to sit back and laugh while reading some of these reviews, as you've quoted I mean YOU KNOW someone is paying HEAVY BUCKS to get some of the praise and "inflated boasts" and "ratings" that gear too often undeservedly gets! And it's always the mass marketed, highly advertised product that apears on 6 pages of some rag that get's a garanteed AAAAAAAA rating or whatever...then slowly manages to slip to class C the next two or three "components" lists and such!
Well I for one have come to expect it, and understand why this kind of thing exists. It's obvious. The industry is relatively small, and needs help from all sources if people are to make good living at this stuff. Infact, audio/video equip manufactures would have anyone picking up a components list guide to believe that sonic and visual Nirvana exists by simply buying their class such and such of product!...and it doesn't. I've heard systems on many many occasions, comprised of all "Class A rated" components sound just aweful! But then all seasoned audiophiles know that you simply can't throw gear into a system and expect wonders. Doesn't work that way. Balance, synergy, set up, acoustics, placment, tweaking, calibrating, knowledge, experience, and dedication goes LIGHT YEARS BEYOND what the average audio enthusiest ends up with, or understands what is capable! And no, the best audio/visual systems don't come from simply reading a list of recommended pieces, maybe hearing them (possibly set up wrong), then buying them online from some questionable warehouse, and sticking em in a room at home with out a clue as to what they're doing! It's no so simple.
So to me I don't mind if a review doesn't say much! I'm used to reviewers "dancing around" the specifics and generalities, or "padding" a review for whatever reasons or incentives/agenda's they might be writing from! I think it's simply "ear candy"! I mean even if it "told it like it is", what is that going to do to the oveall quality of what people end up with who read this stuff anyway? In my experience, little to nothing more than they would end up with from reading "Enquirer-Star-Globe reviews" on audio gear!
This is why I've always laughed at some of these major mag's reviewing systems. IT doesn't really do much more than provide enjoyment, and something to fantasize about! Because you still gotta end up trying this stuff to find out anyway to KNOW ANYTHING of what something does in a REAL SYSTEM, IN A REAL ROOM!
So I say LET EM MAKE THEIR MONEY, and preach there sermons, and put whatever they want in "Class whatever!" In the end, it doesn't matter..let the informed understand what's what, and what's not! The true audio experts, serious audiophiles and those "in the know", will always have better systems, better audio/visual experiences, and the ability to pass on WHAT THEY KNOW to those whom they have good relationships with.
Yep, relationships and money. The Gas prices go up a couple/few times ever year, after the news reports some oil tanker dropped a couple of barrels of crude overboard when they came port-side. Or the water and power companies decide to declare some mysterious and suden "suppy shortage", and prices go sky-ward. Hummmmmmm....I see a trend here.
The idea that I can know how a component sounds by reading about it is absurd. What I look for in a review is a complete description of the unit, features, specs, circuit description, price, and perhaps a discussion of how it compares with similar equipment. I will decide if an audition is worth the trouble, and I will decide if I like how it sounds.
Madmilkman: There are more than a few "underground" audio publications that don't accept advertisements. Several are still in business after many years and others are just starting. An older "independent" is Bound For Sound, which is run by Martin DeWulf. Another "audio upstart" is Richard Hardesty's "Audio Perfectionist". Compared to Bound for Sound, Audio Perfectionist has been around a very short period of time. On top of that, AP is far more costly to subscribe to.

As a side note, many reviewers are not "full time" reviewers and operating costs may not be as high as you had imagined. Most have "regular" jobs and do this for fun and to keep their writing skills honed. This is besides the "perks" that come with the job and / or some spare income here and there. Quite honestly, i think that very few "reviewers" could actually live off of their salary / payroll dividends from writing such articles, but who knows. I sincerely doubt that you'll get any of those that are well known to discuss their income situation with you. Sean
>
Let's see... magazines make roughly 90% of their profits from advertising... how do you plan to make an ad-free magazine then? Charge $30 an issue? Not pay your reviewers? Have no editorial staff?

If you can find a way, more power to you. But magazines would not survive without advertising revenue.
Sbank, I've definately seen some of that -- but, from my experience, the driver behind it is fear of lawsuits. That's the kind of society we've elected to have so we do have to live with the consequences. No business is immune from this threat and I'd hate to see the Gon put out of business because someone won some goofy lawsuit against them.

In my view, that's a different motivation than trying to please every potential advertiser. By the way, the magazines have two primary motivations: 1) Loss of ad dollars and 2) Loss of free evaluation products. Even the ad-free publications or services have to deal with the second threat.
Sbank, Would you consider the comments in this thread positive toward a product or products?

That's why I opted NOT to mention names. :>)
Rosstaman,
Thought we almost had it here, but some have told me offline that their posts containing negative comments about a product have been deleted from this website. Big brother is definitely out there.
But you didn't hear it from me!

Cheers, Spencer
What we need is a non-profit, public service audiophile publication that doesn't depend on advertising dollars from the very manufacturers whose products are reviewed. Unbiased, objective and factual reviews. Good luck! And may the force be with us!

Until then, may the farce be with us, or don't subscribe.
great thread,i dont believe that you can get an honest review to save your life,everything is always light & airy or it has great timbre or some other words put together to sound cool yet staying vague enough so no matter what you think they said there is always a question
(what the hell are they talking about)

its gettin worse too,its spread to some of the dealers,
i went to a new dealer that opened near me,i asked if he carried mcintosh,he didnt answer but pointed me to the krell,i asked again about mcintosh,he said he didnt cary mac,i asked him why,here is his response,

well,mac is good but ya know how some things change but they really stay the same & how mcintosh is still mcintosh but it really isnt,i asked him where he learned to talk like that.

i think as time goes by & the more cash theese reviewers bring in in advertisements it will get far worse,you might as well ask a bum on the street,you never hear a cut & dried review,its never just great or bad its always full of mumbo jumbo double talk & condescending attitudes.

my advice to all is to spend your cash on a good bottle of wine or good smoke if you prefer & leave the audio mags in case you run out of toilet paper.
mike.
I often find there is plenty of (I really don't want to use the word negative) objective information in the body of many articles. What is important is that you don't read the last paragraph where the reviewer endorses the piece of equipment they just subtly ripped. I get virtually all of the rags delivered to my home. Recommeded component issues are worthless. Virtually every piece of equipment reviewed the previous year is recommended. I highly recommend reading some of these mags for their entertainment value. And yes, having to weed through vague, confusing and contradictory comments is annoying.
Another consideration which is somewhat ironic, even though we may consider their reviews "wishy-washy", I have seen these reviews make or break a specific product and/or that product's manufacturer.

Which means, someone MUST be embracing the results of these "reviews".

An ass for every seat or, vice versa. I forget how that goes.

I still remember Chip Sterns rev. on the Mesa Baron. He thought it to be the next best thing to sliced bread. (My dealer 'made me take one home;--- It was much less than my MR 9mk2.) After Chip's rev. and on the same page,--- John A chimes in; "Chip is new and is eaisly pleased". BTW, it was Chip's first article. MR9 mk2 class A Mesa Baron class B.--- No they can't do all our homework for us. Sometimes what they use with what is telling,as well.
There's as much wisdom in this thread before my comments that exceeds that in your typical issue of the big two, both of which I subscribe to. A-gon & audioasylum clearly offer more independent thinkers. Unfortunately, I hear there are also lots of stealth shills among us. Nothing is perfect.
At least with the mags, you know to take it w/grain of salt. When those with vested interests fail to disclose on the boards, it's a real conflict IMHO.
Trust your ears, trust your friends w/good hearing, and trust your gut. Read it all, but a little healthy skepticism never hurts...Cheers, Spencer
Bingo! You hit the nail on the head. The writing is vague and tries to please the manufacturer. I love the interpretations offered above.

On the other hand, I've seen some reviews in Stereophile recently that have surprised me. There are direct comparisons to other units and specific strengths and weaknesses are listed -- e.g., the bass extension is better in unit A but the midrange is more liquid in unit B. Those are the types of reviews that are more meaningful to me. Of course, our reference should always be live music and units don't have to be compared to one another to produce a good review -- however, few products are uniformly better than others. I've heard speakers that reproduced a piano brilliantly but couldn't do much else well. I'm auditioning a cd player now that sounds "hissy" at the high end but is really wonderful in other ways. Sometimes, the direct comparison of two or three different units (e.g., cd players or cartridges) is the best way to produce a meaningful review. I love it when that happens, and it appears to be happening more often now. Let's hope it continues.
Yeah, but it's still ONE person's perception in a system and room that's very different from yours. Even if they could nail down an opinion in no-uncertain straightforward take-no-prisoners honest terms, it still won't help. You have to listen for yourself in your setup. But, you're right - why bother reading the whole article if the terms they use for one piece of equipment has been used in other reviews. I mean, there is a limited vocabulary in how to describe the sound - even though they manage to come up with novel ones. Avoiding being judgmental apparently has made its way into the audio reviewer's world.

What I think would make it interesting is if they created categories of sonic qualities (soundstage, image, dynamics, etc.) and assigned each a number value for the piece in review. Add up all the numbers at the end and divide into the price of the unit. That, I think, would give an instant comparison at a glance.
You are correct sir but in my auditions of many CDPs, preamps, amps, speakers, cables, DACs, etc., I found what I preferred is not always what others preferred. For example Sonus Faber Cremona speakers are to me very musical and more warm sounding than say Talons. Each speaker has their strengths and weaknesses but they are different. In someones system, or musical preference, they may choose one over the other. Same with say a Musical Fidelity Tri-Vista CDP versus a Wadia 861, versus a CEC transport and Audio Logic DAC. Which one should I tell you is better? None, they all have strengths and weaknesses and I cannot say which one you would prefer in your system. Same with say the Joule Electra LA-150 versus the Lamm LL2 Deluxe, one is better on vocals the other is better at dynamics and details.

So your conclusions are correct that the mag reviews don't really say anything anymore and you have to go hear the differences in your system with the music you prefer to make your own decision.

I cancelled my mag delivery a long time ago because of this.

Happy Listening, I mean auditioning!
The turntable was good but phenomenally pricy. The CD player was musical but lacked accuracy and detail. The preamp sounded good but was poorly designed.

I agree. The "i'll tell you everything but won't come out and say it" kind of reviewing is what allows horrid products to look good in a print review. The reviewers hope that the readers are intelligent enough to read between the lines and that the manufacturers aren't. Given that "well reviewed" units that truly are "pieces of junk" end up selling like hotcakes, i would have to say that the manufacturers are happy and that the readers aren't as smart as the writers think they are.

That is either the situation or the reviewers are simply "crafty liars" and "good salesman". Then again, it costs plenty of cash to publish a "glossy rag" and charging $6 - $25 for a subscription isn't going to cover the expenses involved. As such, the reviewers and / or publishers have to make sure that they can cover the bills and that is where "supplementary income" aka advertising revenue comes into play. Stomping on products in print doesn't bring in the supplementary income and then the doors close. Vicious circle and it is why the reviews aren't worth much. That is, unless you are intelligent to know how to read between the lines and / or know how to interpret specs ( if honestly tested and published ). Sometimes, you can't even read between the lines because the product has been so heavily sugar coated in the review that the frosting blurs where one line starts and the other ends. That's when you HAVE to know how to interpret specs. Sean
>
The messages I get from your three excerpts are "good, but very expensive" for the turntable, "OK for the money, but not really good" for the CD player, and "looks good and sounds OK" for the preamp.

Manufacturers are obliged to praise their own products as the best, or the best for the money. They rely on the magazines to position their products in the constellation of the marketplace against direct competitors, often with allusions to the manufacturer's reputation or the designer's history. This almost never involves criticizing the product, just different levels of praise. I find I've become adept at interpreting such bull.

Often price is brought in to help justify a recommendation as a good value, or to assign a product in a not-as-good category. Typically, the more expensive the price, the higher the praise. Perhaps this is real, i.e., better products actually cost more and deserve their praise. It is rare to see a critical review of an expensive item.

Having said all this, I wonder philosophically whether we are seeing some diminishing returns effect on technology. I believe most types of products are better today than they were twenty years ago. Within this context, even a cheap version of something today might be better than something considered pretty good twenty years ago. The same reviewer who was able to make sharp distinctions twenty years ago might find most products sound pretty good today and not too different from competitors. As a reader, I would still like the reviewer to be able to make sharp distinctions today, but it may not be realistic anymore.
I totally agree Buscis2. You hit it right on the head.

I also subscribe to these two unmentionable "journals" along with a couple of the British ones, and, in addition, to several computer music and music production publications coming out of Britain. I am not British, but, pound for pound, after many years of reading this stuff I have to say that the Brits say more with fewer words and get to the point and provide value in the process. I mean they actually tell you what something SOUNDS LIKE! The quality of writing in the US has sunk to new unfathomable lows. It is a sad reflection of our society and of our refusal to recognize the value in our public educational systems. Disqualifer: I have no professional connection to any trade publication, either here or abroad.

Anyone from Agon want to start a better audio publication with me? At least one that raises the quality bar to the level of the Brits? Of course, in the US it's ALL about advertising and marketing. And that's sad because the real function of good marketing is to provide VALUE to CUSTOMERS. Something some US trade publications seem to have forgotten.

Non-compensated reviewing is why we are all here at Agon, where REAL value in better understanding and enjoying our interests takes place, daily.