Equipment reviews, are they credible?

Hi Nrchy

you said> "How do you know which of the above categories the reviewer of the piece of equipment comes from?"

I'm a little hurt because I really feel I have done my part in this regard. My handle was adopted after numerious e-mail requests in the interest of full and fair disclosure. Furthermore, I am on good behavior as I have been told that if I fail one more urine test I will not be allowed to post here anymore.

By the way, the "Bose Lifestyle Acoustamas 5s" sound great and have the added advantage that you can put the subwoofer under the couch with no ill effects and are small enough to cryogenically treat the entire tweeter in your own freezer.

A real value!

Sincerely
I remain,

Your point is well taken. Time will tell, if the reviewer consistently reports things as you hear them, the he or she is a genius.

These are all very valid points. Unless a review from an "unknown" goes into great detail as to other components that have been tried, how the system is configured, if other components or cables were changed to obtain best performance with that specific piece of gear, etc... one should simply consider a "review" a public posting of that persons opinion. The advantage that a well known reviewer has is that they have a "track record" of what they like / don't like. From their past writings, one should be able to decipher whether or not they have similar tastes and preferences and decide whether or not they would agree with the reviewers comments of a product on a regular basis.

If you do find yourself liking similar sounds and products, all the better for you. That reviewer can make your life a little easier by letting him do some of the detective work for you. If you find yourself disagreeing with said reviewer over a course of time, you can use reverse psychology i.e. they did not a specific piece of gear, so you might like it : ) Like anything else, you have to sort through the personal preferences and bias and try to read between the lines.

Some reviewers ( professional and / or amateur ) are quite vocal about what their likes / dislikes are and that is a good thing. I would rather have someone tell me "straight up" that they think something rocks or is junk. Knowing more about their background and how they go about things would tell me how much value i should place in their opinion. None the less, someone that is adamant about a product ( good or bad ) probably has their reasons for coming to the conclusions that they have.

As a case in point, here's a quote from Richard Hardesty, formerly of WideScreen Review and now of "The Audio Perfectionist". He is a reviewer that basically tells you what he likes and why. He will also tell you what he doesn't like and why. If you don't agree with him, that is your prerogative. At least he is honest and tells you what to expect out of his reviews up front.

"These are just my opinions. You may have different tastes in music. You may have different tastes in film. You may have different tastes in a lot of things.. and that includes ( insert product type here ).

You might use my reviews as an aid to help establish your own merit system. Listen for the characteristics that i describe and see if you can hear these things, and then decide just how important each of these sonic properties are to YOUR enjoyment. Modify some of my biases to fit your personality and you'll get what YOU want, not just what sounds good to me."

I think that about sums it up. Most reviews and comments are worth what you pay for them, mine included. If you are interested in reading more of what Hardesty has to say, i'm including a link to his website. Try checking out his free downloadable sample issues AND the "watchdog" series. Keep in mind that you MUST have the latest version of Adobe Acrobat for these files to read properly. I found this out the hard way after pulling my hair out. Sean
>

http://www.audioperfectionist.com

Woo Hoo !!! It is SOOOO nice to have links. THANK YOU Audiogon : ) Sean
>

'"Bose Lifestyle Acoustamas 5s" sound great ' !?!?
- Ohno ! He's failed another test !
Twas good t'know ya !
----------------------------------------

The reviews are of course biased because people buy what they like. It didn't take long to realize this about audioreview reviews. Also most people simply don't have the long term exposure to a variety of components that a professional reviewer does. I think this affects their views.

It is interesting to read why they didn't buy that component's competitition. No reading between the lines there !

Clueless is King!

Opinions are like....uhm, you know....

I have less of a problem with peoples' biases, or even their agendas, then I do with the poor quality of effort and time taken which is being displayed in 90% of these so-called "reviews". A heck of a lot of the responses are more informative and thoughtful than the "reviews"! I at this point frankly think that there is a higher percentage of worthwhile user reviews to be found on Audioreview.com, which is very disappointing to realize.

I believe the easiest way around this situation is for Audiogon to institute a "peer-review" system for the "reviews" themselves. Each review should have, right on the page with it, a box that permits the readers to vote on two catagories:

1) The quality of the review from a standpoint of thoroughness, informativeness, thoughtfulness, how well-written, and how enjoyable it was to read.

2) To what degree they either agreed or disagreed with the writer's findings and opinions.

Both catagories could be represented on a 5-point scale, and the site would automatically compute a running tabulation of the average reactions of Audiogon members to any review. The average result for both catagories could be shown by a pair of decimal numbers between 1 and 5 that would be displayed in the title line of the review, helping browsing A-Gon members to search out quality work, and avoid the dogs.

But more importantly than that last benefit, the review-grading system would have the immediate effect of greatly encouraging review writers to take their time and do good work in the first place, so that it would preemptively weed out all the worthless one-paragraph, unsupported quickie opinion-mongering jobs that are badly written and painful to read (and in my opinion, basically somewhat disrespectful by the "reviewer" of the members such a "review" seeks to be read by).

Without something along these lines being implemented, it's already clear to me that this whole feature is fast turning into mostly a waste of time, and something that will simply begin clogging up the site. To facilitate better reviewing efforts being successfully attempted and completed, however, there are also two things that Audiogon needs to do about the way reviews can be written and posted on the site (I have not yet written a "review" under the new catagory as such, so if I'm about to mention features already in place, please forgive my ignorance):

1) The site needs to be constructed so that a review writer can draft their article over more than one session if need be. The writer must be able to save the current draft of their article on the site, and be able to return to it later to continue writing and editing, submitting the finished work for posting on the forum only when they are satisfied that it is ready. This feature should not be limited in number of sessions or days that it can be used.

2) The writer must further be able to add footnotes or corrections to the body of the original article once it has been completed and already posted as the thread-head, for the life of the review in the archives. These additions should be automatically dated just like a member response below would be, but will have the 'privilege of place' of being located with the original article, for the reviewer to add any pertinent addendums, references, links, corrections, or follow-ups to the review. Reviewer replies to member responses should still be located below with the all the responses, however.

P.S. - Alternatively, we could just send all of our equipment to Clueless for him to review and be done with it. ;-)

The best indicator of credibility is the quality of the review itself.

I think a reader bears the responsibility of considering and working through for him or herself the questions that you've raised. It's no different than reading someone's advice in a thread; you know how to distinguish good, thoughtful advice from the bad.

If you think that people need to be "protected" from bad reviews, then I think this is hopeless. Set up a voting system or whatever, but (1) it's no more credible than the review itself, just adds another layer of opinion on top of opinion, and (2) readers will still filter the data through rose colored glasses anyway.

I'm not suggesting that "reviews" be discontinued. I think they can be of real value. Which is why I began with the confession that I was thinking out loud. It might be a good idea to have them in a seperate catagory instead of where they are now.
Establishing credibility is one of the main issues. The other is establishing a format or something like that to avoid the "Ah reeeally liiiiiked dat new CD ting. It sounds puuurty!"
It seems like too many of the single paragraph reviews don't understand what others may be seeking in their review.
There are people (like Sean or Viridian or Zaikesman, unsound, and obviously clueless) whose comments I enjoy reading. They are generally insightful, or at least funny, which is often more important. The point is once I know someone to a degree their threads or responses have value, people like bishopwill have none.
Maybe the whole thing is getting used to the new item, but it sems like there must be a way to improve the reviews.
For example, I saw a review of my amp a couple of days ago. I am very happy with my amp but the reviewer compared it to an amp I know to be vastly superior and said it was just as good. Again I like my amp, but this isn't a point of opinions, the other amp is far and away better. That is just irresponsible. Maybe this one review is the root for my thread but I've seen other poor quality reviews of equipment I own. The reviews do no one any good.
ARRRRGH, clueless, help me!!!!!!!!!!!!!!

I think personally the reviews are there to encourage Audio sales which is what Audiogon is about.
There is a lot of chat on these forums about abstracts,music,art,hedgehogs,tweaks,DIY whatever and although there is no hostility from the powers that be about that,I think they realise that they would like to see the forums relate more to people spending money on audio equipment.
Hence the flood of reviews which of course are valid on whatever level you place reviews on,some will need more info as to how they were carried out.
For me there has been too many..too quickly...I think reviews are incredibly powerful from whatever source....

>"The site needs to be constructed so that a review writer can draft their article over more than one session if need be."

For now all you need to do is write it in the wordprocessor of your choice and copy and paste.

I remain,

AudiogoN, thanks for weighing in on this discussion. Your post actually helped to clear up many of the questions that initiated this post I hope I was not misunderstood as condemning the reviews. I plan to do some of them in the future. Possibly even of equipment I own, but who knows. In my mind this has clarified some confusion or uncertainty. Keep up the good work. I hope to see some of the changes implemented soon and also to see reviews from some of the people who responded to this thread.

Cluesless: That would imply a having a working knowledge of how to use a computer. ;-)

Turnaround, your point is very well taken, and I was thinking about some things along those lines after I posted above. I think the pertinent difference in this case between Audioreview.com and Audiogon is the way the reviews are set up to browse. At Audioreview.com, you click on a particular brand and model of component, not an individual review, as with Audiogon. This means that at Audioreview.com, you don't in effect wind up wasting your time opening worthless reviews; as you scan the possibly long page of reviews of any one piece of gear, you can pretty quickly tell the wheat from the chaff as you go, and don't have to wait to open and close separate pages, which takes considerable time all together if you want to peruse many reviews. Here at Audiogon, you must open each review individually, so without some external indicator of the review's quality, you often feel gypped for having wasted the time.

But again, I want to stress that the main thrust of my review-ratings idea was to force an improvement in the quality of the articles themselves (it would probably also have the effect of reducing somewhat the total number of reviews posted - not necessarily a bad thing, IMHO). As for this idea's merely being 'opinion on top of opinion', I think it's much easier to have many people agree on the worthwhileness of an article, than on their assessments of audio gear (also not a bad thing).

Audiogon - thanks for pointing out that the feature allowing multiple writing sessions is in place (has anybody used it so far?). I actually suggested this in an older thread from several months ago that touched on these topics. Whether you read that then, and whether it was a novel idea at the time if you did, I don't know. But I'm glad it's here, and I for one will try to take advantage of it sometime soon.

I agree, a listing of reviews would be most helpful. In fact, if you wanted to search out a particular piece, how would you go about it? I was just looking at some of the reviews & when I tried to do a search for them I came up empty.

Zaikesman, I also like audioreview's format for quick scanning through to find the good ones.

As for a rating system, I've seen them work well on some bulletin boards for guaging which threads are worth a look. I've noticed, however, that on Amazon.com high ratings often go to very short but favorable revews. I suspect that it's because people who like the reviewed item want to support to the good reviews, just as people like to write positive reviews to taut their gear. On rollingstone.com, you'll often see an album get three (out of five) stars in the editorial review but five stars in the readers' rating (anyone with a modem connection can rate the album). So it rolls into more of the problem of self-justification that Nrchy pointed to.

I do like your idea about creating incentives for people to write better reviews in the first place. My suggestion would be for audiogon to add questions to the review form like "what equipment did you compare this against?" "what other similar equipment have you owned?" "how long have you owned the piece?" "Name the three greatest weaknesses of this piece?" etc.

These questions would encourage writers to include more information that would help us understand where they are coming from. They could also get the reviewer to confront common questions that a reader would ask as he or she writes. Most importantly, I think it will allow the reader to get a sense of the reviewer. Even if the reviewer enters "none" for weaknesses, you know something's rotten. More disclosure and information means more context for the reader to understand the review. Let the reader be the judge, but it helps the reader to have more information about the reviewer.

Going even further, I would say, don't let reviews be posted unless the author fills in all the blanks. It forces the writer to take some time with the review and discourages hastily dashed, "This is the best CD player ever. The end."

Of course, there will still be reviewers who'll just go enter "N/A" for every blank and write little more than "This is the best CD player ever. The end." And it doesn't stop people who have an agenda from posting. But I think it will be more obvious among the many reviews which writers have taken their time to craft the review an which ones havent'. In other words, we may not cut down the fluff, but it'll be more clear to the reader what is fluff.

Yeah, I think we're on the same page about this stuff, Turnaround. What you pointed out concerning a ratings system's being potentially biased towards favorable reviews of gear anyone rating the review personally happens to like, is why I proposed a two-pronged ratings approach as detailed above. One rating for the quality of the article, one for agreement or disagreement with the take of the reviewer. My idea is that this would tend to keep the raters more objective about article quality. I think your idea of some 'leading questions', as we say in sales, to guide the review writers is also not a bad idea.

P.S. - Nrchy, I'll raise a hand for Bishopwill based on my exchanges with him. You listening, BW? Get in here and defend yourself! :-)

I was away for a while so missed the start of this aspect of the forum topics. I have read many with interest and found some of them useful. So, I think this is a worthwhile addition to the site. However, I would like to echo comments above regarding problems with their integration into Audiogon. Perhaps a separate section for the review postings--or at least for the archived reviews--would be better than including them with the other forum discussions. My 2 cents.

Some reviews beat a stick in the eye. Others do not.

A small point that I particularly enjoyed: Anarchy (nrchy) asking Clueless for help. Beautiful combination!
On subject, reviews are useful IF they include important parametres such as up/down stram equip & reviewer's tastes. I get a lot of insight from posters' opinions (owners or not) on equip when I "know" them somewhat. As long as posters relate their experience and related parametres, reviews are priceless -- be it for NOT considering an item someone else recommends: that person's contribution is VERY important. Agreeing is just as important as disagreeing with another's listening tastes when it comes to obtaining a USER'S information. The latter means, someone who has given serious considering to a piece of equipment -- indeed, has even purchased it!

I for one, always go by USERS EXPERIENCE -- to my detriment perhaps? I choose the opinion of a user with limited product experience over that of a professional reviewer with enormous product experience (I must be getting old).

It's a good idea to put the reviews in their own section. I also agree that many reviews are terribly short and little more than,"This is the best ___ I ever had." That is not a review. I think a lot of the reviews are not reviews, but sweepstakes entries. I would remind reviewers that, while it is nice to win a prize, the purpose of the reviews is to help fellow members learn about various equipment, and its behavior in other's systems. Write the review in a manner that you would like to read yourself, with the info that you would want if you were considering purchasing that piece of equipment. Would you buy that piece based on the subjective opinion of someone you don't know saying "This is the best ____ I ever had"? When he has only had a raft of low-fi, ie.,"This Yamaha is way better than my last Panasonic". I am seeing a lot of names I never saw before, submitting reviews as sweepstakes entries, with informationless reviews of low-fi equipment, in a shameless effort to "stuff" the drawing box. I would like to see a limit of one entry per drawing period, per member. I would also like to see the review moderation stiffened to preclude worthless entries. Perhaps even the addition of a prize for "best review" would increase the quality of the reviews submitted.

Pardon me if this question has been answered before, but is the review section an appropriate place for a 'shoot-out' or is it only for single component reviews? To me, there is a difference between comparing a component to another one that you may have owned or are familiar with and a real shoot-out with component swapping, use of a standard reference disc, etc. in a short time frame.