A Question About Reviewer Techniques


I've been reading the audio rags since the mid-80's and continue to subscribe (albeit with much less enthusiasm) to TAS and Stereophile while also reading the online publications occasionally. I've always wondered why reviewers do not evaluate products in systems recommended by the manufacturers. For example, a recent review of a loudspeaker has generated some controversy because the journals main review system uses speaker cable from a national hardware chain. I'm not criticizing the use of these cables (haven't heard them) but rather the decision to evaluate a particular product (in this case loudspeakers) using cables that are not recommended by the speaker manufacturer. Why don't the review mags simply ask the manufacturer what cables, electronics and source materials will bring out the best in their products and then make an effort to evaluate the product using these components? Any novice knows that system synergy is vitally important in obtaining good sound but somehow this core principal is thrown out the window (for the sake of expediency?) by reviewers who simply pair products under evaluation with whatever happens to be lying around their listening room or with whatever works with their other components. Then they write a review which is somehow supposed to be "authoritative" but which may not show the product in the best possible light. Anyone else find this a little troubling?
128x128dodgealum
Put simply... most publications don't have the resources to be able to buy all their reviewers new systems for every component they review. And since most reviewers are either not paid or not paid a lot, it's probably a little much to expect them to fork out a few grand for a speaker cable that a manufacturer recommends.

But maybe you're very wealthy and could fund all these purchases??? While you're at it, could you kick in a few million extra to build reviewers more acoustically perfect listening rooms (probably several, for different size systems/speakers)?

If you want everything to be perfect then why not open your wallet and fund it? If not, why don't you quit griping about what you get for free or practically free?
You have a point, but there are other considerations. In my opinion, this type of practice would also bias the testing. Furthermore, what would you think if a reviewer is asked by a manufacturer to test and review their new $2,000 speaker, but the manufacturer suggests using cables that cost $20,000 because they really bring out the best in their speaker. I also think that it would be a marketing and financial fiasco for a speaker manufacturer to market their speakers as sounding best only with certain select electronics. They would be out of business in no time at all.
If the speaker manuf. thinks Cable X is the way to go, the speaker should COME WITH a 3 meter set. extra charge for longer, rebate/credit for shorter.

Was the speaker not reviewed properly? Did it get 'panned'?
The author of the thread has a point and I try to demo equipment with other equipment deemed to be synergistic by the manufacturer (particularly because most brick and mortar shops in my experience no longer care about two-channel equipment and do little toward careful set up).

Regarding reviews, it is important for readers to know that they are subjective because there are few if any constants - different ears listening to different equipment in different rooms with different levels of room treatment, different A/C power supplies and different music - scientific the process is not. In light of this, it helps to have familiarity with the various reviewers' respective systems, listening preferences and biases. I think many reviewers are conscientious and I do not intend to disparage anyone, but the only thing I tend to take seriously is Atkinson's measurements.
I wouldn't consider any audio magazine review as "authoritative". Audio magazines are businesses. They make money by selling ad spaces. Their prime objectives are to convince manufacturers to place ads and to promote "high end" to as many people as possible so that they can charge more for the ads. Have you noticed that most of the reviews read like promotional material from marketing department? So keep that in mind when you read them. And no matter what you do, never buy equipment based on reviews alone.
All is solved, system synergy requires the manufacturer to send their component and all other parts, cables too, they want to be used to the review. This way, the system is reviewed. Now, if the manufacturer could just send the 'room' and all its furnishings and treatments, we would have perfect reviews, forever.
Raquel, Atkinson's measurements serve some purpose, maybe for an engineer, but the measurements don't necessarily guarantee or translate into good or bad sound.
Cyclonicman:

Among other things, speaker measurements indicate voltage sensitivity and impedance / phase angles so I know whether my amps will drive a speaker. Information about cabinet vibrations and frequency response plots tell me if the speaker may be overly colored. Frequency response plots also tell me whether a speaker's bass performance may excite room modes in my room. Measurements of tweeter dispersion tell me whether a speaker has to be listened to on the tweeter axis in order to get even high-frequency response. The waterfall plot can reveal a dirty tweeter. Etc., etc.

With amps, measurements reveal actual input impedance so a person can know whether the amp, when used with his/her preamp, will have bass rolloff issues. Wattage measurements into varying impedances reveal whether the amp can drive low impedance speakers (which is most speakers). Amp measurements usually include discussion of whether the circuit uses feedback and if yes, where in the circuit feedback is used.

Preamp measurements tell me whether a preamp's voltage gain is so high as to limit the useful range of its volume control (one of my amps is unusually sensitive and can be driven to high output with little gain from preamps featuring high gain). Preamp measurements also show output impedances at various frequencies, again allowing me to see whether there may be mismatches with my amps.

In short, measurements can be very helpful and speak directly to performance in your system if you know what to look for.

Regarding your other comments, I'm not aware of a manufacturer of $2k-level speakers pushing their use with $20k speaker cables - I'm not saying that the practice doesn't exist, but it doesn't make sense. I do, however, know of speaker manufacturers that speak well of specific cabling due to perceived synergies (e.g., Wilson likes Transparent and uses it inside their speakers). This is a common practice and good business to the extent that equipment/cable synergies are crucial to top performance.
IMO, the key is for the reviewer to have a consistent system that they are very familiar with, so when the reviewed component or speakers are placed into the system, the reviewer can accurately describe the sonic result, and also make comparisons with other similar equipment they have heard in the same system. The reviewer should also describe any anomalies with the system or room and how those impact the review.

What I would have issue with is, if a reviewer were to place equipment in a system that does not match electrically (e.g., impedance mismatches) or if the review rig in general is not well-suited for the equipment being reviewed, such as reviewing a Krell amp (known for world class bass) with small monitor speakers (known for limited bass below 60hz); or reviewing large speakers, made to produce high spl's in large rooms with an extended powerful bass response, and putting them in a small room. Duh, no wonder it doesn't sound good. I think the wire is small potatoes as long as the reviewer is familiar with their system, and as long as the impedances, efficiencies, room and system characteristics are appropriate for the equipment under review.

Finally, no review is the final word, but simply a subjective opinion based on the reviewer's experience with other equipment and listening preferences.
Raquel, what I was trying to convey is that once you have your measurements, you still have to walk through the wonderland of equipment and attempt to put together a synergistic system. The measurements are a starting point and there will be many choices that should be "right" because the specs say so, but they don't guarantee a synergistic system. The point about the $20,000 cables on the $2,000 speakers was meant to be absurd!
>> The point about the $20,000 cables on the $2,000 speakers was meant to be absurd!<<

A terrific point but unfortunately these situations exist. Look at systems such as joeyboynj who has a very average set of components and thousands of dollars in cables.

This occurs more than you imagine.