What should be mandatory in every professional published review-
When testing a company's newest amp, preamp, etc, and it is a refinement of a prior product that was on the market, ie, a Mark II, an SE version, a .2 etc, it should be mandatory that the review includes a direct comparison with the immediate predecessor. IMHO, it's not enough to know ion the product is good; it's also important to know if there is a meaningful difference with the immediate predecessor.
I'm fan of Pass Labs, and I just looked at a review of an XP22 preamp. I find it very disturbing that there was no direct comparison between the XP22 and the XP20. And this lack of direct comparison is ubiquitous in hi-end published reviews, across all brands of gear tested. I don't blame the gear manufacturers, but rather the publications as I view this as an abdication of journalistic integrity.
nothing should be mandatory in a (professional) review. zero. nada.
other than stuff like editing for effective and entertaining writing.
i want no intrusion from any entity regarding review content. i want it up to me, the reader, to judge the merits. if the organization can do measurements then fine, but that’s no plus or minus to me one way or the other. relating measurements to actual listening is just not any slam dunk. but this is a very contentious issue.
reviews are data points that can be helpful, entertaining, or a complete waste of time.
i do appreciate everyone who does reviews and the time and effort it involves. i value the print HiFi magazines and on line HiFi review websites. our hobby is better that they exist.
Amen to that. Fastest way to ruin reviews, demand what they must and must not be. This is probably a tough one to explain, but people's opinions are really only valid to the extent they are the sincere opinion of the listener/writer. The more you let anything intrude on that, the more you are reading about the influence and not the genuine impression. In other words, if you are gonna dictate what must be in a review, go get the gear, listen, and write your own review. Probably be just as good if not better. And then you are in the game, not Monday morning quarterbacking.
Honest comparisons against speakers, amps, preamps, of similar caliber is my answer. Previous version of the same brand previous model or other brands of similar perceived quality.
This would be almost impossible to implement. Most equipment is loaned by the manufacturer. The reviewer won't have a component from two years ago, sitting around. The manufacturer won't be interested in shipping old product out for review. Nice idea, but unlikely.
Telling people what they have to write doen't ring true with me.
An unintended consequence of the internet is that people have figured out how to use it to poison competitors and pump up their own equipment, often in the form of "reviews".
Unfortuntely readers are often not smart enough to figure this out.
Not Mandatory....but....some things I would like to see because they would be helpful to potential buyers:
1. Take your own photos...top, bottom and back...and obviously the front and if it is speakers, grill on and grill off...why the bottom...I might care about the feet or the holes, or anything else that can be seen.
2. If its an amp, then use you IR gun and shoot the amp in a bunch of locations...how hot does it get near the bottom, side, back...maybe I'm using a somewhat closed in cabinet and I care if it reads 95 degrees on the top but 120 on the bottom left.
3. Show me another photo or two of the packaging.
4. And if the article/review doesn't have room for the photos, no problem, post them elsewhere and give me a linkl
5. What are the problems that might bug someone...does it click, or pop, or hum or anything on start up or shut down?
6. The sound...the more comparisons the better....and don't just use "audiophile" music....use some stuff that people actually listen to beside audiophile music.
7. Say a few words about your own bias...what do you like, what do you not like.
"Mandatory" was not the best word to use. Short of plagiarism, libel, slander and the like, reviewers are free to write in any fashion they wish. If they write in a manner that readers like, they'll be rewarded with a devoted following and continued work. If their work is meaningless to the majority of readers, they'll find themselves looking for a different line of work. And, lots of reviewers are in the middle, with some readers loving their work, other not, and a healthy contingent in the middle who are "OK" but maybe not thrilled.
If one is a long time reader of a magazine, you'll have your own mental list of reviewers to which you pay close attention and others which you ignore or discount. And, that's not to say the ones you like shouldn't be open to suggestions to improve, though as several have noted above in this thread, sometimes practicalities get in the way. How often, for example, does the reviewer have the prior model sitting side-by-side with the new one under review? If not, that means that any comparison would be from memory, and probably have been in a system with completely different companion components. Such a comparison would make for interesting banter, but hardly a solid reference point.
..it should be mandatory that the review includes a direct comparison with the immediate predecessor. IMHO, it’s not enough to know ion the product is good; it’s also important to know if there is a meaningful difference with the immediate predecessor.
If it was, and they did there’d be no point in doing reviews.
You only have to compare a 1980 NAD 3020 with a 2022 NAD 316BEE (whatever) to see just how much of what the reviews continue to imply, ie a solid 40 years of continuous onwards and upwards improvements, have actually achieved.
Professional reviews aren't necessary, because they really defy all purposes of reviews. By definition, a professional review is something that a professional makes living on.
Reviews must not be professional. They have to be taken from casual consumer
yeah sure like the reviewer has one just sitting there to compare and why would they do that at all. They are there to promote the industry not say well the older model was almost as good for a few grand less. How would that help a manufacturer at all?
BTW if you think reviewers make a living at this stuff you are wrong!
If the word "mandatory" is freaking people out how about each publication developing its own "template" that their stable of reviewers would adhere to in order to insure some continuity and comprehensiveness to formally reviewed products. If I were the editor of such a magazine or online publication, I would suggest the following:
Room dimensions along with any room treatments;
Associated equipment;
Statement of preferences--as an audiophile, what am I shooting for in terms of sonics;
Relationship with manufacturer--any financial interest, discount offered on purchases, history with brand, etc.
Seems to me this would help the buying public make better sense of the reviewers findings.
would be nice, but I agree, completely impractical/impossible to always have the previous unit on hand for comparison...and then, there are so many other products it would be great to compare it to...not just the XP20 but an ARC, a CJ, JRDG etc.
I would place the responsibility on the manufacturer to explain/demonstrate to their clients what is different between the former version and an upgraded or updated version.
I place the responsibility on the reviewer to,
At a minimum, use partnering equipment that would reasonably be expected to be used with the review equipment - i.e., no mis-matches, and to
Compare the reviewed equipment with comparable competitors available in the market at the time of the review.
I was fortunate to have seen reviews by the same reviewer, of my new DAC and of the same manufacturer’s former DAC. The direct comparisons were certainly helpful in knowing what I should expect. However, that will mostly not be practical, as several here have pointed out. 6moons usually does a good job of making meaningful comparisons, IMO.
We get paid to write reviews, we are willing to hump lotsa different gear in and out of our systems and spend several hours ferreting out all the details of a component and how it compares to something else (unless you write for TAS), and we’ve had the advantage of the perspective of hearing a lot of equipment in our own rooms and systems. Plus, we can write well enough to both convey thoughts and concepts in a way that people actually like to read.
That out of the way, personally if a manufacturer would go to the trouble/expense of shipping a current and former version of a review product I would’ve been happy to write about the differences (I actually did that once). But, that’s rarely offered and would greatly increase the cost to the manufacturer with shipping costs and such. But, and much more doable and what Soundstage.com always required, was that any reviewer had to at least have a comparable piece in their review system so there was at least one applicable point of reference. Anyway, trying to do what you’re asking would fall mainly to the manufacturers and not the review publications unless a reviewer happened to have the previous iteration of a component, which happens but is relatively rare.
One major practical issue is that ow the reviewer needs to be familiar with the previous unit. Or, now his/her work is at least doubled, to in effect, review two products. In the end, most will not really want that review. They want to hear about something totally new to them.
And regulating reviews? Yikes. what kind of shirt ought i wear if i undertake that task? Is a tennis shirt ok? Or is a music Hawaiian required? Hard to say....
I took this a question of what one wants in a review not exactly a mandatory edict. It’s not “communist” yet! Assuming most of the peeps here are Euro or American.
i stopped reading stereophile because i got tired of the lack of detail in their speaker [listener] reviews. oh sure there is a detailed tech section, but i wanted the reviewer to tell me some basic user info, such as if the speaker is on the mellow or harsh side of neutral, how well the bass section reproduces organ pedal notes, the quality of stereophonic imaging esp. off to the side, the clarity of specific frequency ranges, dynamic limitations, et al. what i got instead was descriptions of how a given system played obscure records in their collection that are not likely to be easily found, this didn't tell me how the speaker would likely sound with more general kinds of music in my listening room.
here’s the issue. very few of the reviewers you know make their living reviewing. They are published, yes. Might even get a few bucks for a piece. but they have real jobs. Its a hobby
@itsjustmeThat’s absolutely true, and it certainly was with me. But it’s also true that we’ve gotten to hear lots of gear in our own systems and in our own rooms so we have lots of experience in a real apples-to-apples comparison environment that we know well. That gives us a big leg up on people who usually only get to hear equipment in varied systems and rooms that introduces far too many variables combined with rapidly-fading aural memory that severely taints the audition and decision process. Any piece I reviewed I had a comparable piece to compare it to in my review system. Plus, every reviewer needs to be able to express in words the things they hear in ways that prospective buyers can relate to and use to help make more informed purchase decisions. Deride us as just hobbyists if you will, but we’re all in the same boat as you and most of us just want to provide useful information to others in the hope it could possibly be helpful. And trust me, we ain’t in it for the money cause we’d mostly all starve if that was the case. My advice — find reviewers who you trust and seem to be real and use them for useful information, because their impressions can be extremely helpful especially in this world of disappearing dealerships. Just my $0.02 FWIW.
@emrofsemanonId encourage you to read reviews from Soundstage.com. We were always required to do a thorough review of all the basics you’re looking for and also be able to compare it to a component in a similar class. It’s a no BS site, and I was always able to write whatever the hell I heard with no outside influence or editorializing. It’s a straight-up and honest publication. FWIW.
Just in case it was not clear, no disrespect at all. No ding on "professionalism". but folks were talking as if they coudl define high end reviewers like plumbers, lawyers or economists. Heck, mybe behind screen names we know each other, but many of the top reviewers that i happen to know have very different professions. Investment banking, engineering, defense analysts (seriously!).
It would be desirable to have a comparison with the previous iteration of each piece of equipment reviewed, but that would mean you'd have to own the earlier iteration of every piece of equipment you review. I think any reviewer who does have access to the earlier version of equipment under review would naturally compare them, and I've seen quite a few reviews that do such comparisons. But I think buyers who are thinking of upgrading from Mark I to Mark II of a piece of equipment would be a rather small part of the interested readership. It's far more useful to compare the piece at hand with its competitors.
They don't want to compare their newest stuff to the older stuff because then people would realize there is almost never any difference in how it sounds. They change things cosmetically, add in some marketing buzz words, and voila! Oh, and raise the price, the real goal.
So - If you are looking for industry pricing and write nice-nice ask to review…Reviewers love equipment more than music - analytics versus emotion
@stsYou couldn’t be more wrong. Go ahead and ask any publication to write a review and see how that goes. Then actually go through the whole review process from humping the boxes into your room, unboxing it and setting it up in your system, go through the burn-in process, evaluate the product in the context of several different genres of songs and then compare it to something competitive only to realize many of the things you “thought” you knew were in fact incorrect and then listen all over again. Then get all the product specs, measurements, etc. together and the write all that up together with a comprehensive, insightful, and interesting piece that’d hopefully be useful for someone to read. Then box the thing back up and hump it to UPS or FedEx to have it sent back to the manufacturer. Until you’re willing to do that, don’t make it sound as if it’s like doing nothing to get dealer pricing. Trust me, on a per-hour basis you’re better off working at a diner. And I know plenty of fellow reviewers and to a person it all starts with a love of music, but we happen to also like to find equipment that makes that music more involving and sound its best. You know nothing of what you speak. Go ahead — I dare you to try to write a professional review and see how that works out for you. Until you do that, I’d suggest you keep your completely ignorant and uninformed opinions to yourself. Thanks.
First, do you believe any of the reviews done by the magazines? They have been so biased for decades and even getting worse. When was the last time you have seen a negative review? Never! Most reviews end in “the best I have had in my system”. I’m just elevating what other people have said in other posts that I agree with.
As for comparing to a prior release, they use their memory or notes which you can’t or shouldn’t do.
I would like to see a real review when they compare like components and they rate them best, 2nd best, 3rd best, etc.. Exactly what occurs in other hobby magazines.
For example, most car mags have no issues giving a $300k car a 2nd place rating. There is always a better product wether you are using your ears or your hands and legs.
Where's the OP gone? What should every poster here, if they were professional, do? There's a myriad of preferences and likes/dislikes, just as in any marketplace, but @mikelavignesummed it up very nicely. You choose to read what you are looking for, since we don't have journalist and publications police... thankfully. Maybe another source can take on the role of industry critic's critic... mind you the marketplace here and elsewhere seems to do this very well already.
A few other things should be mandatory: What does the item smell like? How many total parts does it have? Can it survive being attacked by gorillas? Does the warranty cover excessive rust? How long can it float? Can its lights lights trigger a seizure? Can you sit on it? What temperature does it reach when disconnected? Without these things being addressed you just can't take a review seriously.
snapsc- fully agreed & of course what ancillary equipment was used & what other equipment was tried w/ the reviewed piece. Stereophile is good w/ this, Absolute Sound often not.
The Stereophile comment is a good one. One of the main benefits of being able to post your system here is anyone who wonders what kind of a listener you are can go and see for themselves. If you want to make one thing mandatory for reviewers, this might be it. Let them write and say whatever they want. Long as we can look and see what was what when they formed their impressions. It's the audiophile version of the Rosetta Stone
Would Seem the General consensus is Research yourself, listen to your own hands on hearing. Not all reviewers are bloated blowhards that said who cares it’s you that has to be happy and that’s what matters. IMO we’re all Professionals when it comes to liking what you hear, no course involved with that, Don’t have to train a monkey to eat a banana, why should we pay a b…hard to tell us what they hear. Now if it’s specification technical that has value, but it still boils down to your personal hearing.
I’ll take it one step further. Specifically, Speakers. But the same could be applied to amplifiers and other electronics.
I would like to see pictures of the actual drivers used compared to the predecessors or other speaker makers in the same price range. That is not just external pictures but the actual drivers themselves showing their magnet motor structure compared to others. For that matter, seeing the crossover inwards would also be helpful.
Some of the upper end loudspeaker's go for close or above $100K. I know the cabinets are expensive, but what makes up all the other costs. R&D? I dunno.
But it would be helpful to know what makes these products better than their predecessors or competitors.
With other electronics, show the circuit boards and the types of materials used. Why is it better from previous designs?
From the pictures that are available, some of the new components look to have very little inside heavy metal chassis.
I dislike the clickbait on Youtube--especially when a reviewer says "This could be my last _______ (speaker, amp, DAC etc...) Then 3 months later, they're reviewing another item that replaced their last whatever.
at this point it is safe to say that most reviews are flawed and not an undeniable reference for purchasing anything. they are constrained from mentioning anything remotely detracting about the sound and in fact barely describe the sound character at all. they simply cannot bite the hand that feeds them.
most of the text in a review has nothing to do with how it sounds. they are mostly fluff that includes company history, the new technology and why it should sound better, room and system setup, and maybe a little about how it sounds playing often obscure music. even if the peice receives an enthusiastic recommendation there is little to go on in terms of how it will complement the sound of your system or its overall sonic character.
buy from a place that allows a good return window and listen before you buy if possible.
You must have a verified phone number and physical address in order to post in the Audiogon Forums. Please return to Audiogon.com and complete this step. If you have any questions please contact Support.
