2. misc-audio
  3. 189510-20253

Measurement Bias

Measurement bias is the idea that if you know the measurements of something that information will already bias your perceptions of it’s performance. For instance, knowing the g-force at which car A might slip vs. car B could affect your perception. B, having a higher g-force rating should be a "better" car but if you did not know this you might rate A as better. It may be more fun to drive.

I’ve seen this happen in a review in (I believe) TAS. The review was for a DAC I purchased. The reviewer noted it as "noisy." I read this after I had purchased and been listening to the DAC for a while. I was more than a little shocked, I could hear no noise whatsoever from the DAC. I pulled out an oscilloscope and sure enough, there was unexpected ultrasonic noise on the outputs. I eventually did sell the DAC, but not because it was at all noisy, but because new DACs handled Redbook (44.1kHz/16 bit) tracks so much better. Nowhere in the review did the reviewer mention they had measured it, and they certainly did not point out the deficiencies in Redbook playback, but the reviewer absolutely presented this DAC as noisy but otherwise OK.

So, my point is, that making assessments on the experience that comes from a measurement in audio is tricky business, and if the reviewer is aware of the measurements ahead of time it will absolutely bias them into hearing things which they’d otherwise not, and leave them blind to other real world challenges.

If you want to put together desirability with measurements you need to look at the work Floyd Toole or Bose have done and others in this area and you wont’ find it in a frequency response chart. Of course, Bose’s research is proprietary, but absolutely no one one earth has spent more money on assessing value vs. measurements and manufacturing dollar than Bose.

PS - Please don't argue the quality of Bose speakers here.  I'm not arguing for or against them.  I'm arguing that the research done in tying together desirability and engineering direction is outstanding.  That is all.

erik_squires

Showing 1 response by mijostyn

mijostyn's avatar
mijostyn
8,223 posts
 

One of the reasons this endeavor is so entertaining is that there are no standards for what a stereo system should sound like. It would be impossible to make them. The same speaker in three different rooms is three different sounds. Three different types of speaker in the same room are three different sounds. It is up to every single one of us to tailor our systems to sound the way we want in the room we have available within financial constraints. I personally believe principles are more important than measurements. Some measurements are more important than others, but given the range of issues related to speaker performance in situ, specifications and test measurements reside in the background.

I have said on multiple occasions, what someone says any piece of equipment sounds like means nothing to me because I have no way to interpret this. I only care what I think something, usually a loudspeaker sounds like in my room. When it comes to loudspeakers the specs are virtually meaningless, but the design speaks volumes. In many instances the specs are misleading. 

The important measurements to make are those you can make in your own listening room in regards to amplitude, group delay and dispersion. If you can't hear distortion any measurement is of no significance. If you can't hear noise any noise level is of no significance. 

 

         

More to discover