Audio Science Review = "The better the measurement, the better the sound" philosophy

We have talked about this before. The fact that he pans some equipment without listening to it is enough for me to discount his opinion.

You’re an audio theologist with money to spare, not a scientist.

Thank you, yes I am. A scientist knows that a tomato is a fruit. The food artist knows not to put a tomato in a fruit salad. 

There is an art to putting together a great system. The person guided only by measurements will often create an unlistenable system with each piece of gear having outstanding measurements. At least that has been my experience.  

@amir_asr  given the huge premium they charge for their amplifiers.

How is price part of audio science? 

I will answer: you need to understand electronic design.

I don't need to know how to build an amp to listen to how it sounds any more than I need to know how to build a violin to know how it sounds. You understand electronic design and measurements, most here understand the art of audio reproduction of sound, it's more complicated than just making a measurement. 

Long term listening is still the gold standard of audio evaluation. Short term evaluations often lead to errors but it's much harder to hold an inaccurate evaluation over the course of many listening sessions and sometimes it takes many hours to find the faults of the playback system. 

@djones51 I thought our aural memory was very short. 

I understand that to be true. I'm not a fan of A-B testing. I think you have to optimize each setup and listen for a long time. In the end you'll be able to learn which system you prefer but I wouldn't say you'll know which one is better, too many variables. Unless there are gross differences.

Note that I listen to every speaker I review.  Measurements are the foundation but I test them by listening. And equalization.

I appreciate your time here, we won't ever agree but you have explained your process in detail and that is more than most would do. But I have a question about the quote above. You added you listen to loudspeakers "and equalization", can you explain that please. I'm assuming you equalize the loudspeaker for your listening session?  Is that right? 

That is a myth unfortunately.

The myth is that short term blind testing with unknown recordings, with unknown equipment, in a strange environment provide accurate results. Blind testing generally gives a null result, it's use as a scientific tool is very questionable. It generally gives no result. 

Thanks for the reply, Amir

@invalid  "You know what blind ... tests gave us..."  

Yes, Bose. Dr. Bose's graduate thesis was blind testing and removing frequencies his subjects didn't hear were missing. 

One positive side-effect is that others with the same speaker can apply that EQ and see if it improves their listening experience. 

I understand, thanks for the detailed explanation. I see that as work for a speaker designer but not for a reviewer. If it helps you, OK, I get it. 

If I had to describe my place in this division, I would say I’m looking at a much bigger picture. I work with the end product, the sound that comes out of my loudspeakers. I’m working for a sound that pleases me. Mr. Amir is looking at the microscopic details, the grain of the wood, if you will, and I’m assembling an entire landscape. The tools I use are different than the tools he uses. 

If you get sick, you can go to a doctor...

Just wanted to mention a Doctor of Medicine is an Arts degree, not a science degree.  

@decooney "Most of the musical sounding components I prefer, don’t measure perfectly."

That is the whole point of this forum. The measurement guys are convinced, that better measurements equal better sound. Except better measurements just equal better measurements. Their favorite dig is: "Enjoy your distortion". Well, I do, thank you very much. 

@kokakolia  "

I am the same way. The sound which pleases me comes from a single fullrange driver mounted on a transmission line and powered by a tube amplifier. That's what I like. I listen to a lot of chill music and vocals are the most important quality for me..."   

We have a passion for the art of recorded music reproduction. We are feeling the emotional connection as we glance at a masters painting. Amir is analyzing the paint pigment and is missing the bigger picture. This hobby is big enough to let everybody join and share their experience. There is no need to exclude other opinions nor any reason to provide measurements to back that opinion up. I guess the thing that bugs me is some think I need to provide documentation of my opinions. I most certainly do not need to prove anything to others. The sound that comes out of my system is the only proof I need. 

@thyname Yes really....

Well, that explains a lot. I think I get it now. They are in search of the perfect measurement at bargain basement prices. That's why my opinion carries no weight, they are not seeking better sound, the foundation of being an audiophile. Thanks that really sorts it out for me and why I found their viewpoints so perplexing. 

"The contention is if you take a " less expensive " component with better measurements can you differentiate it from a more expensive boutique component with worse measurements or an SS one from tube using controlled testing."

I exclude pricing completely, it has nothing to do with sound quality. Why include it? It's just somebodies' opinion of a components worth. It's an opinion of value, not science. It has nothing to do with the sound I hear coming from the product. Pricing is a subject for Comsumer Reports, not equipment reviews. It certainly is not part of "Audio Science". Subjective reviews are the place to include value judgements. We all like a product that returns good value, but the "science" of audio quality should be devoid of any reference to price.  

@djones51 ...as a measurement guy "better sound "  is subjective and not anything I pay attention to. 

Really? You do not pursue better sound? Are you pursuing better measurements? What is your interest in audio if it's not better sound? I'm confused. 

I like the guys that have stated that science is observation and measurement. Someday maybe we will have the tools that allow our observations to be measured in a reliable and repeatable way. 

...and I guess my next question to Amir would be: Do you have any significant data to show the correlation of SQ of components priced from $1000 to $30000 as an example. 

Dang it guys. Any electromechanical device (loudspeakers/phono cartridges) will break-in. How long does that take? Beats me. Then we have electronic components. They need to burn-in to reach their final electrical state. How long does that take? Ask the manufacturer. And finally, we have cables. Do they need to be burned-in? I don't know but if somebody says it took X hours for a new cable to sound right, I have no reason not to believe them regardless of what the mechanism might be, even if it's their brain adjusting to the new sound. So, what's the problem that needs to be disputed? 

@amir_asr  "I am not in audio science.  I am posting here.  Ask a pedantic question, you receive an answer in kind.  :)"

You have been very generous with your time, I'm sorry I am testing your patience, but I've am seriously curious how you inject price or value judgements into your reviews.

@amir_asr " At the risk of stating the obvious, measurements don’t care what something costs."       

Thanks for the reply. BTW: I do look at the measurements you post on your website and read the forums occasionally. I don’t think I ever joined though. I could see that my experience would not be valued so I don’t look at the website as much as I look at others. If the environment allowed my input, I would be much more likely to use the website more. I don’t want to make a wrong guess but I’m thinking if this thread was on ASR, I’d be long gone. Thanx again for joining in the discussion, I’ll lay back now and watch on the sidelines. No need to respond, you have been very patient with me. Thanx.

The problem with measurements above all, is the logistics. Take loudspeakers for example. We can't drag a loudspeaker home and run a Klippel sudo-anechoic measurement on them before we buy them (even if that system could provide useful information on panel loudspeakers). So, even the measurement guys do what we all do. They go down to the shop and listen to the speaker and buy the one they like. 

I’m running out of popcorn and the mudwrestling match shows no signs of a clear winner in sight. 

We all won. Everybody got to express their opinion and some readers may have learned a few things or at least been given some food for thought. ASR closed their thread while we are still up and running. I'm not too proud of the personal attacks but very happy this thread stayed open despite the contentiousness. 

If I had to sum up why websites like ASR are not that interesting to me;

I would say firstly, they have an emphasis on low price gear, a segment I don't often shop in.

They have one mantra: If it's not measured, it's BS. Totally ignoring the scientific area of observation.

I am called an audiofool. 

Distinct bias against people of means.

Almost total lack of statement level equipment testing, unless it's something that measures poorly.

Little time actually spent listening and quantifying what is heard.

No respect for outside opinions. 

Websites like ASR could double their viewership if their approach allowed discussion and opinions outside their own belief system, but they could still present their rebuttal with their own emphasis on measured performance.  But not with the you're a crazy, audiofool, with sight and general biases that make your opinion worthless.  

I'm not sure what is yours and what is quoted text. Use the quote function. 

"Once more: listening tests are the gold standard in audio research."

Amir repeated this a few times but he means controlled statistically valid listening tests, that are well beyond the individual listener (Since one reference point has no statistical validity).

"Importantly, don’t confuse creation of art with replay of it."

We don't have the science that will create a perfect playback system so there is some art in creating a pleasing system. It may not please you or your measurements but at long as it pleases the end user, the goal is accomplished. 

Except we as humans can’t integrate all of that meaningfully to get all the way there in terms of predictive sonics, so often we have surprises when we listen....and use both measurements and subjective listening to assemble gear that gives us enjoyable sound.

At least somebody is honest about consuming all that data. I've been looking at that data for many decades, it's useful to a point, then it gets too much to process. 

They have one mantra: If it’s not measured, it’s BS.

@russ69 how many times will you and others repeat this lie?

 

...or a USB cable has no effect on the output as Amir had shown in tests,

Does that not include a measurement to underscore any result?

And yet it's the ASR folks are called closed minded and "dogmatists."

Oh the ironing....

Your thread on ASR was closed. Who has the open mind again? 

I am pleased to know that your business in high-end audiophile gear is good. I think that it is important for everyone to know this for future reference.

I'm sure this is sarcasm but unsure how you think that is funny? 

Nobody cares except the rapidly failing high end components bricks and mortar market.  And the site we are currently on - enjoy it while you can.

The high-end market is very strong with more choices than we have ever had. There is a shift away from the brick and mortar business model but online and direct sales are booming! Business is good.  

Stop feeding the ASR Trolls. 

You are right, I've made my points, nothing more needs to be said. 

So knowing that our senses are so misleading how can the audio subjectivists rely solely on their hearing? 

The bottom line in audio is how the system sounds. Some of us have multiple decades of experience putting systems together. If I sit down to listen, the only thing that is important is the sound coming out of the loudspeakers. I don't need to have my choices validated by data. Measurements are a semi-interesting side bar discussion at best. 

but has always reminded me of a bunch of parrots in a cage where one parrot has only learned what he has heard the other parrots squawking.  

The AVS forum has the same herd mentality, might even be worse than the ASR forums. At least the ASR has tons of measurements to look at.... 

I can't stay away. The thing that makes interesting reading is, interesting writing. Writers make good reading, not engineers. Writers that use their vast vocabulary and emotions to explain what they are hearing. That is helpful, although not 100% reliable. On the other hand, the guys on ASR can't say what something sounds like until they perform double blind tests with their control sample of buddies, not very useful. That is why their subjective reviews are one or two lines of not very descriptive words. 

@henry53 "...an audio product sounds bad based solely on a THD of 0.003% compared to another product with 0.001% ..."

@crymeanaudioriver "...Haven't we had enough hyperbole and lying for one topic already?..."

Correct me if I'm wrong but THD is part of the SINAD formula, I think. So wouldn't a higher THD number put it further down the list on the SINAD chart. Isn't the whole idea of SINAD to rate sound quality? 

When you test an amplifier, you have to pick a condition that all amplifiers will pass, or the data can't be used for comparison. But all loudspeaker loads are not the same. Hook an amplifier up to an Apogee with 1 ohm or less impedance and you will smoke a lot of amplifiers at high power levels. Measurement testing is not the final answer. Real world testing is final arbitrator. 

Negating the value of measurements because they only apply in 99.99% of situations is a fools outcome.

In the segment I shop in, I'm very likely to get a difficult to drive speaker. I can list them if you would like. My history shows about 80% difficult loads.

"The Apogees and MBL speakers are anomalies whereas common Maggies just need power..."

I guess it all started with the Infinity Servo Static and then the entire Infinity Quantum reference line...and my little MMGs just didn't like my AES (Cary) Six-Pacs, did weird things, actually buzzed at low volumes? 

So you are saying Stereophile receives gifts...

Not sure you know how the industry works, but there are many perks, depending on who you are. Firstly, equipment might be "on loan" with no set date of return or there may be a heavily discounted accommodation price if you wish to keep the item you tested. It is also known that the distributor/manufacturer might "wine and dine" a reviewer when delivering a piece of gear. And last but not least, a reviewer might be flown to the factory, for a tour and several day accommodations and meals. These are normal sales perks in some industries and certainly true in audio.  

"...to find the one obscure situation where ASR measurements may not be correct, and attempt to use that as an excuse to discard all the work they do. Maybe that fools some people..."

I'm not trying to discount anybody's work or opinion. I'm just relating my experience. ASR doesn't really spend much time testing quality high-power amps or gear priced, let me be kind here, above the most modest level, just a few pieces. I like the measurements provided by ASR, but not much gear that I am interested in appears on ASR, so it's not particularly useful to me.  

@russ69 , please do list them. I am curious. The Scintilla seems unique.

Reading far more knowledgeable people on this topic, my 99.99% is much closer to the truth than 85% - 90%.

You travel in different circles than I did/do. Yes, they sell more cheap bookshelf speakers than large difficult to drive loudspeakers but that doesn't mean the hard to drive loudspeakers are an anomaly. Back in the late 60s there were not many high-power amps for home audio. Let's start with the Bose 901s series I. They sucked the few 200 wpc amps dry. So much so that Bose built the 1801 amp to solve the issue. After the Infinity Servo Static system and the SS1A was being developed (a difficult electrostatic load), Infinity developed the Quantum line reference line of loudspeakers. They were amp killers and started the development of many new high-power amps as did the Magnepan Tympani Loudspeaker. Amps like the Ampzilla, BGW, Phase Linear 700/400, and others. Later the Infinity Reference Standard 2.5s/4.5s, RS1Bs, Monitor IIAs, stacked Advents. You might say the Magnepans are not hard to drive but a pair of 3.7s driven at volume can stress most amps. Probably not as many hard to drive speakers by pure numbers today due to the requirements of multi-channel and low power AVRs but there are still many out there in the high-end world.

Haha, the argument would never end. 😅

The science is settled on that issue, vanilla is America's most popular flavor.

Question for the ASR guys. Referring to the SINAD chart, it is labeled "Excellent, Good/Very good, Fair, and Poor".  That's in reference to its measured performance but still perhaps a value judgement but if you are measuring noise and distortion with some made up formula. Would not it be more valuable if the groups were labeled; "Audible, Perhaps Audible, and Not Audible"? 

If you are of the "ASR" state of mind, you start of by immediately acknowledging our fallibility... 

So, it's a seven-step program like Alcoholics Anonymous? 

Minus the religious proselytizing of AA.  For some I suppose it could be a similar detox from being marinated in audiophile myths ;-) 

No religion, you just have to believe in a higher power than yourself...maybe like Amir? 

While ASR-ers don’t actually perform DBT or similar nearly as often as they demand it of others

If a DBT is done, you have to count each test, no matter the number of participants, as a single data point. Because the test results may be biased. So, the number of tests required to be statistically significant goes well past the majority of tests ever reported. Make the sample of tests large enough to be statistically significant and you are at the man on the street level of participants, not trained listeners. And that is how you get Bose level of audio quality. 

I didn't watch the video, but I'll take any Arnie Nudell designed Infinity over any Toole tested loudspeaker.

@russ69 +1, Nudell on trickle down tech:

I was very lucky. As a young audiophile, I visited every stereo shop in SoCal listening for the shop that had the best sound. I found it at Woodland Stereo, the shop that helped Arnie develop the Servo Static and other loudspeakers. I got to listen to the products and hear about the problems. By far the best sound in SoCal. I didn’t need a DBT to hear the difference, they were head and shoulders above the rest. Arnie had the education (Physics) and the ear to make extraordinary sounding loudspeakers. And I got a glimpse of it all as the products were tested and improved at the shop that I bought my gear from.