Audio Science Review = "The better the measurement, the better the sound" philosophy

td_dayton,

Thanks, and your points are well taken.

As I mentioned in my first post in this thread, I am thankful for the approach and information available at ASR, which doesn't mean I share precisely the same goals in terms of my own equipment purchases.

For instance, I love the Devore O/96 speakers.   When they came out there was quite a vitriolic comment thread in Stereophile with some self-claimed experts declaring the design ridiculous and incompetent "nobody who knew what he was doing would match that size woofer with that tweeter" etc.

And yet I found, like so many others, that the O/96 was one of the most special, beguiling sounding speakers I've ever heard!  So given there is a sort of "direction" to what is evaluated as "good" or "bad" design on ASR (again, well justified IF you adopt the underlying goals), that kind of product mostly won't be on the radar there, and indeed the Devore speakers have been mocked by some there.  In that sense, ASR isn't the place where I would likely have been led to auditioning those speakers.   It was the subjective reviews, and listening reports from other audiophiles that got me interested, and I found the general take to be bang on when I heard those speakers.

This is one example for why I have often defended the usefulness FOR ME (and many other audiophiles) of subjective reports about some audio gear on the ASR site. Exchanging subjective impressions is not as reliable information per se as objective measurements, especially when you have a specific goal for the measurements (correlated to the sound you want).    But, as I argue, that doesn't make them completely useless or always inaccurate. 

And I find the case interesting in regard to the research on speaker design (Toole/Olive/Harman Kardon etc).  The research suggests that, in the blind tests, I would be most likely to prefer Revel speakers over the Devore, as the Revels are successfully designed to hit the "preference" target that arose out of blind testing.

I completely accept that research.  I think that if I were to go to the Harman Kardon facilities and engage in blind tests, it would be safer to bet my money I'd select the Revel speakers under those conditions.

So what do I do with the fact that I actually did audition a few Revel speakers (which were very competent sounding as predicted) and yet still heavily preferred the Devore?   Well, it could be that I happen to be one of the outliers, and even in blind testing I'd select the Devores.  Less likely, but possible. 

My own decision would be to purchase the Devores over the Revel based on my auditions.  This is because, in "sighted" auditions even IF there are other non-audio factors influencing my perception of the Devores being more engaging, those are the conditions under which I'll be listening to the speakers.  If there are other factors influencing my perception of the sound (e.g. the looks), fine, I'll take 'em because it's sure working to keep me engaged!   Plus, this approach has led to plenty of satisfying purchases over the years, and I just really, really enjoy listening to all sorts of different loudspeakers, so I could never be one of the "order it just on measurements" folks.   But I totally GET that an emphasis on measurements work for some other people.  (AND, btw, over at ASR most members would prefer to hear a speaker before purchasing.  Even when you've narrowed the field down to several "good measuring speakers" there's still enough variation to bring in personal preference).

@td_dayton ,

My background is medicine/medical research. Fortunately, or unfortunately, at a biological level we are all rather similar. If we were not, medicine, disease, injuries, would be an even harsher problem to attack. Our preferences will obviously have a combination of genetic, environmental, and experience aspect. That is not controversial. That genetic aspect is driven by evolution and while there are variances, there also strong underlying similarities. We see that in all aspects of sensory stimulus, whether sound, site, smell, or taste.  That is why companies like Walmart pick their color schemes as they are, why food products with multiple flavors always have favorites, why we prefer yellowy light when it is dim, why we like the feeling of soft/plush textures.

Our similarities are even more similar at the hardware level. It is surprising how little variation, when we are young and everything is in tip top condition, there is. That applies to site, hearing, taste, even touch. The resolution, sensitivity, and dynamic range of all our senses varies only by small amounts within a population. Some are gifted with better processing capability and the ability to extract more useful information, but they have similar underlying limitations.

Change a few pronouns, descriptors and names to the above post and all of a sudden it's a diatribe against what he's saying.

All the best,
Nonoise

I do not comprehend the analogy of listening loud and shouting at low volume levels. I am certainly not a genius but have 2 BAs, JD, MPA etc and took physics courses at UCLA. My wife was a bio-chem major at Stoneybrook and has very deep comprehension of mathematics. We are not uneducated "noobs."

I have not claimed to be an arbiter of all things audio, or for almost anything audio. You have assigned that job to yourself.  That you don't understand the analogy of listening to loud music and not being able to hear regular conversation, in the context of your post, is not surprising. Being educated does not mean that you are applying your intelligence all the time. You made two comments about the dynamic range of hearing. One from my knowledge is accurate, maybe even a low estimate. The other was ludicrous. That you do not understand my comment means you do not know how to temper the information you believe you know about hearing with the act of listening to music.

I previously commented there are two paths to take with information. I said when that information conflicts with what we believe, but it also applies when that information supports what we believe. We can reject that information without consideration, even becoming angry, we can accept that information, even without knowing if accurate or valid, or we can research and learn further and try to understand the nature of that data. If the data supports what you already believe, you may tend towards confirmation sources that believe the same thing, however unqualified.

You put down Amir because he is saying things you do not want to believe, meanwhile lauding Darko who is saying what you want to believe. There is no doubt at all in my mind, that where this topic is concerned, Amir is far more knowledgeable and would be far more recognized as an expert by others with real expertise. 

There are a lot of put downs of engineers on audio forums. My background is medicine/medical research. I am very familiar with this mindset from lay people. Curiously, I almost never ran into that mindset from people who worked in unrelated science fields. It was unique to those who worked outside the sciences. Not exclusively, but the majority. Perhaps those who work in the sciences have better training to identify likely correct information from likely incorrect information? They are absolutely skeptical people, but they temper that skepticism with research.

@prof appreciate the thoughtful reply. to your point about studies re: general preferences, there is a massive, ongoing replication crisis in the soft sciences to contend with. this doesn’t mean that all studies are bad - far from it - only that "there is research which shows that..." is not a silver bullet. i hasten to add that (much of) physics does not appear to have this problem. but there are a great many studies which have been proven impossible to reproduce, so i approach this sort of "studies show that more people prefer strawberries to peaches" reasoning with caution. human motivation is enormously complex and the factors which lead person x to prefer A over B can (and very often does) vary at the individual level.

I want to thank the Audiogon moderator for permitting this forum to continue and expose the narrowmindedness of the "objectivist" measurement is the gold standard for determining audio equipment quality.  The ASR site/Amir has not extended a mutual openness to permitting our members from participating on their rebuttal forum or their site. 

@laoman I listened to the entire Darko audio. Wow, a balanced and intelligent conversation. Thank you so much for shedding light on the topic of equipment testing.

@russ69 I really doubt that my listening room, as high quality as it is, has much in common to an anechoic chamber and probably it is good that it isn’t. The former is good for testing, the latter for listening. Thanks.

@tonywinga Yes, trying to prove a negative. Amir is a classless act which he has proven on this site. Schooling, not educating.

@crymeanaudioriver You can’t stop yourself as the arbiter of all audio knowledge. My reference to my wife was clearly that she has become accustomed to listening to the sound of my system over the decades and now enjoys it immensely. She is brilliant and has memorized the lyrics to at least 1000 pop/rock songs. I do not comprehend the analogy of listening loud and shouting at low volume levels. I am certainly not a genius but have 2 BAs, JD, MPA etc and took physics courses at UCLA. My wife was a bio-chem major at Stoneybrook and has very deep comprehension of mathematics. We are not uneducated "noobs." It took a long road to obtain high end sound as I had mid-fi sound for most of my life.

I am constantly learning, it is my nature and in my religion. You are a fool with your analysis/accusation.

Yes, I noticed Amir has shut down his opposition site and apparently has vanished from this board. That is his nature. (Although I must admit I have spent more time than I had anticipated on this topic, due to a great part for Amir and his minions coming to this site to basically attack us and Audiogon).

the assumption that there is a fundamentally rational, measurable basis for emotional responses to an experience (like listening to music) is flawed.

Scientists study human preferences all the time. Social and cognitive sciences for instance.  General trends in preferences really can be quantified.

That doesn't mean anyone has to personally care, of course.

the act of measuring gear is not a threat or a problem for anybody.

It shouldn't be.  But in a certain audiophile paradigm - My Ears Do Not Lie! - people do feel threatened and triggered if someone else appeals to measurements to adjudicate their claims.   The irony is that it is so often claimed it's the "objectivists" who are the dogmatists, when it's often the Golden Ears who are most unmoving and vitriolic in their beliefs.  As quite a few posts in this thread attest.

where i disagree is when people insist that i shouldn't trust my senses when i am determining whether a piece of equipment is good for my system.  reviewers with a subjective bent don't bother me at all, for the same reason - seems blandly obvious that what sounds good in john darko's room might sound like junk in mine. how do i figure it out? i buy the thing and return (or sell it) if it sucks!

I think we always need to keep in mind specific claims about gear.  Speakers, for instance, I think Amir would see "good measurements" - as determined by all sorts of research about what tends to sound good to a majority of people - is just good knowledge to have, and a good starting point.  It doesn't say a speaker will absolutely work for you in your room, but if you have the measurements and you know the character/size of your room and where you'll be placing the speakers, the measurements can indicate what type of problems you might encounter...or not.

A "well designed" speaker will tend to be easier to sit in more rooms than one that is less "well designed" and could be more finicky.  It's not Absolute Knowledge of course, but it's up to any individual what to do with it.  Some ASR members have a good enough grasp of the measurements and their room to be confident in buying based on measurements...and they have successfully done so.  Other audiophiles may be less knowledgeable or experienced understanding the technical stuff, or even if they do understand it, may still feel they always want to audition a speaker first.  (I'm in that camp).  So there is no "dogma" being enforced about this.  It's just offering knowledge for those who can or want to use it.

Then there are all the dubious claims in the audiophile world - e.g. expensive USB cables, switchers and on and on.  In this case the debate isn't so much about "choice" as to "whether the device actually does what it's claimed to do." 

Of course the Golden Ear approach is to just listen and decide for yourself.  Nobody is putting a stop to that.  But for those who want more reliable information - the type of tests and measurements Amir provides is often very enlightening, and helps many people make advised decisions as to where to spend their money.

So a Golden Ear may say "I'm happy I spent $2,000 on my new USB cable, because I think it made a difference" whereas I and many others will look to the plausibility of such claims, as explained by experts.  I find the evidence Amir provides in many of his reviews - the combination of technical theory with the tests in support of the conclusions - to be much more cogent than audiophile anecdotes, especially when it comes to the tweakier side of the hobby.

YMMV...

I enjoy WBF because they do have industry experts posting there. 

@djones51 i appreciate that. 

i became interested in this general topic due to ASR guys on reddit telling me that i had no idea what i was talking about when all i said was that i generally prefer vinyl to digital and that it sounds better to me. "you are an idiot, digital is objectively better" etc. like, ok buddy. that's your opinion. have had many other similar exchanges over the past couple years in various forums. i simply don't agree that better spec (however defined) = better sound in all cases, and i think that there are limits to what testing and measurements can actually tell us, for the reasons i offer in my previous post. 

where i disagree is when people insist that i shouldn't trust my senses when i am determining whether a piece of equipment is good for my system. 

I don't think anyone insists you shouldn't trust your senses. I certainly don't.

I could (and should) learn a thing or two from him on how to engage the absolutists and dogmatics one encounters along the way.

The amount of "projection" in this thread is really something ;-)

I’m running out of popcorn and the mudwrestling match shows no signs of a clear winner in sight. 

We all won. Everybody got to express their opinion and some readers may have learned a few things or at least been given some food for thought. ASR closed their thread while we are still up and running. I'm not too proud of the personal attacks but very happy this thread stayed open despite the contentiousness. 

the assumption that there is a fundamentally rational, measurable basis for emotional responses to an experience (like listening to music) is flawed. no one denies that there is ample research and data to back up assertions about engineering, and everyone is aware that there are results from controlled listening tests that indicate the preferences of the majority (of test participants anyway). my view is that figuring out "what sounds good" is something different. life is not a controlled listening experiment, and our individual experiences with music are shaped by circumstances which cannot be generalized, let alone replicated. 

the act of measuring gear is not a threat or a problem for anybody. i agree with amir that more information about a product is always a good thing. where i disagree is when people insist that i shouldn't trust my senses when i am determining whether a piece of equipment is good for my system.  reviewers with a subjective bent don't bother me at all, for the same reason - seems blandly obvious that what sounds good in john darko's room might sound like junk in mine. how do i figure it out? i buy the thing and return (or sell it) if it sucks!

i've done this with topping gear which measures great on ASR. some people love topping - i don't say they're wrong or have bad hearing or only care about specs. my $100 schiit dac simply sounded better and the topping gear was ultimately not worth the expense. why? i don't know! and at the end of the day, i don't care. will i spend another thousand or two bucks trying to better the schiit? probably. i like trying new things and seeing if i can hear the differences - it's fun

The problem with measurements above all, is the logistics. Take loudspeakers for example. We can't drag a loudspeaker home and run a Klippel sudo-anechoic measurement on them before we buy them (even if that system could provide useful information on panel loudspeakers). So, even the measurement guys do what we all do. They go down to the shop and listen to the speaker and buy the one they like. 

@laoman ,

Props for that link to the Darko exchange with Amir. It spoke volumes. Darko is such a class act. I could (and should) learn a thing or two from him on how to engage the absolutists and dogmatics one encounters along the way.

All the best,
Nonoise

I’m running out of popcorn and the mudwrestling match shows no signs of a clear winner in sight. 

Amir is trying to prove a negative. That has never before been accomplished.  
Nothing to see here. 

@fleschler 

"None of those changes is massive but, with even average human hearing having a 100 decibel range from the lowest sound we can hear to the loudest sound we can bear without injury, our ears have a single-scale resolution range of 1 to 1 BILLION—far greater than any known test instrument—so even differences that might seem prohibitively small may be clearly audible."

Point 1:  I encourage you to learn enough about audio, science, or math to check the potential validity of what you wrote.  Point 2:  When I have the music turned up loud and I am engrossed in the music, my wife needs to shout to get my attention. I just quickly Googled that normal talking is about 60db. That means I am struggling to hear 60db. Hopefully that means something to you as it relates to what you wrote.

Perhaps it is my background in a science related field, but I find these concepts rather simple to understand, but without that understanding, I could see myself susceptible to believing things that are not true. Couple that with a belief you have nothing to learn and you become susceptible to both external suggestion and internal misdirection.

Good for him but Darko uses different measurements and often reveals different answers. Choices and variable methods to conduct experiments occur in measurements.

I looked through Darko's site. It was obviously not the first time I was there. I failed to find one measurement except in his discussion of bass decay. Are you referring to his ears as measurement devices?

@laoman ,

Regarding Amir’s comments here. Frankly I find them offensive. He has come along with the attitude of "Let me educate you." I do not need him or anyone else to educate me on what I like and dislike. Nor to I need to be lectured that I do not understand the Science. There are many people far more experienced and qualified than Amir of whom I prefer to take note.

Can you tell us these people who are far more experienced and qualified? I am serious, I would appreciate a list.

I don't think Amir is at all lecturing anyone on what they like or don't like. That would be obtuse and Amir does not appear to be obtuse. Do you need to be lectured on the science?  To enjoy music? No.  To continue to contribute in a valuable fashion on audio forums without being incensed by what others have written and misquoting what they have said?  I think for many on this forum, that should be self evident. For many, when they read something on a forum that does not agree with their world view their first inclination is to think, "you are wrong, wrong, wrong". Maybe we all do that. It is the next step that defines you. You can get on that forum and say you are wrong, wrong wrong, and likely be the one that is wrong, or you can open up Google, and start learning. Proving yourself wrong is much less contentious than someone else doing it, and it is ultimately more satisfying.

@chopandchange , your post is nothing but insult attempting to look like high reasoning and deep insight. It is a tired trope and best left inside your head and not put on a forum. I could as easily say that the posters on this forum are incensed with Amir because he has shown, with hard to refute measurements, that you can achieve perfect recreation of an audio signal with a Topping DAC and a £500 amplifier.

Good post chopandchange. You are probably accurate. I also think Amir has fled. He has closed the thread on ASR which was intended as an insulting rebuttal to this thread.
It is a pity he has gone. I was curious as to how he would answer the comment that his testing methodology on the Chord M scaler was faulted. He consigned a whole product to the dustbin based on faulty methodology.

I personally think the ASR mob are looking to ’justify’ spending much less money to get a hi-end sound.

They lean on ASR for reassurance that a Topping DAC and £500 amplifier is all they need to spend in order to get the very best sound possible.

More than likely most of them do not have deep wallets so perhaps somewhat ’envy’ those that can spend the money to - in reality - get far better sound than measured perfection can really give you.

I do not have a deep wallet to enjoy hi-end at its best - so I could easily been seen as one being suckered into believing that measured sound is the best sound to save money. That would suit me just fine - if only that were true.

ASR Frgirard Your points are

The willful ignorance of the primacy of speaker-room coupling by taking refuge in the magic of equalization.

the willful ignorance of the primacy of the decay in the bass and the rest of the spectrum.

the spending of thousands of dollars on equipment to listen in rotten rooms, a total lack of rationality here and on AG.

I spent $150K+ building my listening room from the ground up (12" 3000 psi steel reinforced slab, 16" multi-dimensional walls with chambered activated carbon bass trap filters, etc). I use no equalization. I have various room treatments for mid and high frequency absorption, smoothing, diffusion and decay/reverb. In doing so, I do not require state of the art equipment to obtain a very high quality sound reproduction absent state of the art/expensive high end equipment. Putting well designed speakers of moderate price (even $1K) will react superbly in my new room. I’ve built several other sound rooms in prior homes but they were good but not great. Then again, I have heard a few superb large systems in very large untreated rooms at audio shows at high cost ($1m+). The room is generally very important (as it is in my live recordings) but sometimes the equipment can perform well in a wide variety of settings (Von Schweikerts).
 

What’s Best Forum I have for several years joined and enjoyed camaraderie of other audiophiles forums. That site is generally more congenial and well mannered than any other forum site I frequent. Disagreements do not involve character assassination or defamation. (Except with posters experiences with Acoustic Fields company/owner). I prefer hearing about others experiences than reading about the gold standard measurements as Amir prefers. Good for him but Darko uses different measurements and often reveals different answers. Choices and variable methods to conduct experiments occur in measurements. So Amir is not the greatest tester of audio equipment. There are serious drawbacks on measurements themselves. If measurements are the gold standard, then listening in a system in a room is the platinum standard.

Audioshark, Audiokarma and older sites I visited were quite nice and amiable. DIYers also was very informative lacking acrimonious postings. Why is that?

@laoman @russ69 Thank you for your experiences which reveal more about Amir and his buddies who accompanied him to Audiogon to TELL us what we need to make intelligent equipment decisions.  Just like you laoman, I don't need someone to educate me at 66 after my extensive listening experiences, what sounds best to me. 

When I upgraded my digital cable two months ago, I've had audiophiles and non-audiophiles just revel in the digital reproduced music.  The audiophiles say "best ever sound."  Well, I can't say that because I have heard absolutely fantastic sound systems better than mine (and expensive).   No problem as I'm living within my means.  My wife, a very tough customer who says she is now always concentrates on the sound before listening to the music much to her consternation, just sat and listened for several hours (also unusual) to a variety of smooth jazz and her rock music (1970s to heavy metal) when I upgraded the cable.  She said it sounds like vinyl.   

Today, Positive Feedback has an article by Roger Skoff-Hi-Fi Weather? Roger Skoff Writes About Something You May Not Have Thought Of…

I found the end of the article not dealing with weather’s affect on sound propagation most enlightening (especially if true) "None of those changes is massive but, with even average human hearing having a 100 decibel range from the lowest sound we can hear to the loudest sound we can bear without injury, our ears have a single-scale resolution range of 1 to 1 BILLION—far greater than any known test instrument—so even differences that might seem prohibitively small may be clearly audible."

Perhaps this link is something all people who posted on this site should look at,
 

@amir_asr

It was a spin and a debating stunt which I called you on.

The part I asked you to respond to was your statement that my "takeaway" after amplifier listening was contrary to audio (and psycho-acoustic) research. The points I re-iterated were quite sound, but I was interested in your counterpoint (as opposed to your talking points).

What you did was elevate the listening test to something it is not, then complain that it doesn’t follow the extensive protocol Dr. Toole used for research. That was improper and I responded to you as such.

Relax mate, I’m not trying to trick you with wicked (sorry, "improper") sophistry. Nor am I running a comprehensive analysis of ASR test methodology, with or without "elevation". No need for all the mansplaining.

The simple point of my earlier post is that we can listen to gear, with some experience and awareness of the pitfalls, without always following the strictest of protocols and still glean meaningful information. I described doing it upthread, you do it yourself with loudspeakers (and you say you think it has value).

Another debating stunt. I do not run a "business" to have a model.

Is there anything to be gained from this semantic quibbling? I’m sure you know that in English "business" has several meanings, including "an activity one is engaged in" and isn’t restricted to commercial activity. If you don’t like the semantics of "business model" just think of it as "modus operandi" or "general approach". You know what I mean. Argue the substance.

And it is not like you have shown any of those editors that hold on to gear perform comparative blind testing of speakers. They have the time according to you but waste it away with who knows what. You want to complain about something, complain about that.

Pure whataboutism. How about those reviewers, eh? Come on.

Of all the comments I have read in this thread, this by djones astounded me:

“This is wrong, as a measurement guy "better sound " is subjective and not anything I pay attention to. “

I thought I misinterpreted the comment, but then

No idea if it is better or worse? Seriously? This explains the reply I got on ASR when I asked “What does it sound like”. I was told that doesn’t matter because it measures perfectly. Well let us take one example. I listen to a lot of equipment and have done many comparisons together with friends, at shows and also in demonstrations by companies. My major complaint about a dac that ASR rates as the best measuring dec is that it sounds like crap. I stated that the mid range is shrill, particularly when dealing with female voices. I was categorically told that I was wrong. I am not wrong – for me it sounds shrill. I was thrown off the site for this.

I would not use this dac if it were given to me gratis. Fine, I accept that others may like it, however I do not. Interestingly only one of the 8 that was present at the comparison thought it was satisfactory. We did not know what the brand was at the time. Perhaps the fact that we are all interested in classical music was a deciding factor.

Another example is an amp that ASR wets themselves over as the best measuring amplifier of all time; it is actually quite good, especially for the price, but there are far better amplifiers for my taste.

Regarding Amir’s comments here. Frankly I find them offensive. He has come along with the attitude of "Let me educate you." I do not need him or anyone else to educate me on what I like and dislike. Nor to I need to be lectured that I do not understand the Science. There are many people far more experienced and qualified than Amir of whom I prefer to take note.

Having done various blind tests over the years, it's a very powerful lesson.  It's too bad many audiophiles haven't experienced their 'sighted' impressions dissolving away when they can't use their knowledge of which piece of gear is actually playing.   There's nothing that sinks in like an actual experience.

(And many audiophiles see blind testing as almost synonymous with "detecting no sonic differences."  Where in fact plenty of differences have been detected in blind tests for various things.  That is of course one way various codecs were arrived at.  And I've had some positive results for identifying differences in my own blind tests).

Another story. :)

Our audiophile society was invited to a stereophile reviewer's home to see and experience his setup (he just left that organization).  He had a new amp to review against his own.  Room was too small for all of us to go in there so we divided into two groups.  First group went in and came out.  Without telling us anything, we went in there.  At the end, reviewer asked which amplifier sounded better which folks did.

When we came out to join the larger group, we realized the first group had voted the exact opposite!!!  The reviewer said he had played the amps in reverse order for them vs us.

You see how faulty sighted evaluation is?  

We create controls in listening tests to create reliable outcomes and avoid the above.

@amir_asr " At the risk of stating the obvious, measurements don’t care what something costs."       

Thanks for the reply. BTW: I do look at the measurements you post on your website and read the forums occasionally. I don’t think I ever joined though. I could see that my experience would not be valued so I don’t look at the website as much as I look at others. If the environment allowed my input, I would be much more likely to use the website more. I don’t want to make a wrong guess but I’m thinking if this thread was on ASR, I’d be long gone. Thanx again for joining in the discussion, I’ll lay back now and watch on the sidelines. No need to respond, you have been very patient with me. Thanx.

@fleschler 

As a cable beta tester, I hear the cable raw, then burn it in for 24 hours.  I can't say I can always tell if it sounds better, but I can always tell when it sounds worse.  Depends on the cable. Doesn't matter to ASR. 

Now do the same thing blind, run a camera and repeat 10 times.  Let us know if you can tell the raw from burned in cable.  Should be easy for you to run such a test.  I post the video on how to do it.  All of us at ASR would love to see such an experiment.  Your anecdotal claim above where you included your eyes and full knowledge of what is being tested, not such much.

@axo1989 

I’m not positioning you as "against" Toole, simply stating that you ignore that specific recommendation. Ignore in the sense of "intentionally disregard" which is what you’ve specifically stated here. If the semantics don’t suit you, change ignore to "doesn’t follow".

It was a spin and a debating stunt which I called you on.  Dr. Toole's collective research is how a specific set of measurements highly predict listener preference.  Those measurements are created by me in every speaker review (and then some).  This is the lead and core of my review.  

My listening tests and EQ are a supplement that I have chosen to include.  A ton of people have argued against it on ASR.  I have answered them in the video I produced.  It is not important or core to my review of a speaker although I personally value it.

What you did was elevate the listening test to something it is not, then complain that it doesn't follow the extensive protocol Dr. Toole used for research.  That was improper and I responded to you as such.

Your business model is based on rapid testing and fast turnover and of course that has advantages and disadvantages. 

Another debating stunt.  I do not run a "business" to have a model.  I have a hobby which creates great value for large swath of audiophiles and the audio industry in general. That hobby is based on objective data on audio gear and explaining the science and engineering of audio.  

As an engineer, I try to optimize for the resources I have.  A $100,000 speaker measurement system needs to be in constant use to provide that level of value.  Me sitting on a single speaker to test for weeks and months doesn't provide the right value.  Creating predictive measurements absolutely does.

And it is not like you have shown any of those editors that hold on to gear perform comparative blind testing of speakers.  They have the time according to you but waste it away with who knows what.  You want to complain about something, complain about that.

@russ69 

You have been very generous with your time, I'm sorry I am testing your patience, but I've am seriously curious how you inject price or value judgements into your reviews.

Thank you.  As a general rule, I do not incorporate price.  If a piece of equipment performs superbly, I give it my highest recommendation irrespective of price.

There are situations where I bend the rule.  For example, in amplifiers that cost less than $100, I allow impairments that I would not in more expensive gear.

Conversely, if something is super expense and performs middle of the road or lower, it will get my scorn.

All in all, it is a personal opinion which can be discarded and reliable measurements and other data used for yourself.  At the risk of stating the obvious, measurements don't care what something costs.

relevant to the topic at hand (as well as the initial confusion between darko and ASR outlooks) - here john darko interviews paul barton of psb, who explains the importance of measurements and listening tests (following toole’s methodology) to his process, as well as the subjective choices he has to make as he finalizes his designs. fascinating stuff, confirms much of what both "camps" are saying:

@amir_asr

To position this as me against Dr. Toole’s teachings is very much out of line. Nothing remotely is true in that regard. I simply don’t have the resources or time to do this kind of testing on every speaker that lands here.

I’m not positioning you as "against" Toole, simply stating that you ignore that specific recommendation. Ignore in the sense of "intentionally disregard" which is what you’ve specifically stated here. If the semantics don’t suit you, change ignore to "doesn’t follow".

Your business model is based on rapid testing and fast turnover and of course that has advantages and disadvantages. As this thread started by conflating ASR with Darko, let’s consider that he takes some weeks to listen to and test a speaker. Now—leaving aside potential flouncing about objectivist vs subjectivist approaches, relative value and the like, please—you could take that time, but you choose not to. Perfectly valid decision (and you’d test far fewer speakers). But it’s your turnover preference that precludes turntable testing, or other comparative methods. So ASR has both value, and weaknesses. We appear to agree.

I highly encourage such efforts. I provided speakers for the second phase above to the organizer and happy to do so for anyone who likes to conduct them.

The turntable will be interesting (albeit just a handful of posts so far, and no progress since March?)

I once asked my doctor if he could help me with research into weaknesses of blind testing. He just about threw me out of his office!

I’m not sure about the value of your anecdote about your doctor. I’d prefer you addressed my question about your allegations against my earlier post, for example.

Finally, thanks for the invitation to post at ASR, I’d never thought of that. 🙄😂😉

My listening tests in reviews is provided on "as is" basis. I do them because if I didn't, I would get more complaints.  "Oh, he doesn't listen."  I have tried to make more sense out of them by developing the EQ technique.  The outcome there has been quite positive with many trying my EQ profiles and liking them over stock performance.  If folks want to ignore them -- and many do -- it is no skin of my nose.  I perform them because I am curious myself how the measurements translate into sound and a form of listening training.

That all makes sense to me.  Given the range of audiophile viewpoints you can't please everyone.  If you don't listen, you'll get complaints about that.  If you do listen, you'll get complaints that you aren't using a rigorous enough protocol.  If you use a rigorous blind protocol, you'll get pushback from the anti-blind-test faction who think blind tests obscure results and you should have listened "in relaxed sighted conditions, like a normal audiophile."

Personally, I think that yeah, blind listening to speakers would hue most consistently to the ASR remit.  But your compromise of  "here are my impressions take them or leave them" seems a reasonable compromise.

@tonywinga 

Some of the greatest sounding musical instruments, in fact most all musical instruments  and even concert halls were designed and built without computers and electronic analysis equipment.  They were designed and built by artisans with skilled hands and ears.  It's all about what we hear.  When someone tries telling me I am not hearing what I think I am hearing, well that goes over with me about like putting a tax on a child's piggy bank.

Once more:  listening tests are the gold standard in audio research.  No one is telling you to substitute measurements for it.  

What we say is that don't go believing marketing claims that have no verification with controlled testing, or make sense at engineering level.  We prove the latter with measurements.  Company claims the power conditioner lowers your audio system noise?  Well, we measure that.  If the result is that noise has not changed one bit, then you know the claim was wrong. 

Why is this so odd for the few of you to accept?  You say your local water is making you sick?  Folks come out and measure to see what is in it.  If it is pure and clean, then that is very important information. 

Importantly, don't confuse creation of art with replay of it.  Our business is the latter. The two are completely different universes.  Audio equipment should NOT be in the business of creating or modifying art.  If it is, then it is not high fidelity.  And will impart the same signature on every music you play -- something I dislike dearly.

As to what you think you are hearing, that is NOT in doubt.  What is in doubt is what you say it means when you did not block all other senses than your ear.

@amir_asr  "I am not in audio science.  I am posting here.  Ask a pedantic question, you receive an answer in kind.  :)"

You have been very generous with your time, I'm sorry I am testing your patience, but I've am seriously curious how you inject price or value judgements into your reviews.

@axo1989 

One the one hand, Amir does ignore Toole’s advice that speakers should be listened to comparatively for evaluation to be meaningful. Even something as simple as setting up a curtain and turntable and enlisting helpers from his cohort of followers is dismissed.

This was a planned activity from day one that I started to test speakers.  Indeed, i have held on to a mountain of speakers for this very purpose.  But you may have heard of a thing called the pandemic.  Our local audiophile group where I was hoping to conduct such tests stopped meeting (and went virtual) so the project is on hold.  Meanwhile, one member did post such a comparison: 

And a much more sophisticated one using a turntable was created as well:

I highly encourage such efforts.  I provided speakers for the second phase above to the organizer and happy to do so for anyone who likes to conduct them.

Such testing is extremely time consuming.  But good news is that anyone can do it.  You don't need my experience or instrumentation.  So no one should be waiting on me for it.  My time is best spent providing objective data such as measurements.

To position this as me against Dr. Toole's teachings is very much out of line.  Nothing remotely is true in that regard. I simply don't have the resources or time to do this kind of testing on every speaker that lands here.  

My listening tests in reviews is provided on "as is" basis. I do them because if I didn't, I would get more complaints.  "Oh, he doesn't listen."  I have tried to make more sense out of them by developing the EQ technique.  The outcome there has been quite positive with many trying my EQ profiles and liking them over stock performance.  If folks want to ignore them -- and many do -- it is no skin of my nose.  I perform them because I am curious myself how the measurements translate into sound and a form of listening training.

@axo1989 

But when it comes to amplifiers, ASR rarely listens, with the general justification that magnitude of differences are too small to differentiate (with some exceptions). As you can see in this thread all listening (that isn’t done by ASR with partial protocols) is routinely dismissed as sighted bias, expectation bias, focus bias etc (you too can play bias Whac-A-Mole). 

The pushback like that is made when the claimed sighted tests go counter to solid body of engineering and research.  Say a power cable improves the sound because you swapped one for the other and proof of being right is "I have been an audiophile for 30 years" and you rightly get strong pushback.  Don't offer such as proof point and you are generally fine.  And even if folks object, you should be cool because you weren't going to prove anything.

How else do you want us to behave?  I once asked my doctor if he could help me with research into weaknesses of blind testing.  He just about threw me out of his office!  He said, "Amir, the foundation of what I do is based on blind testing; I can't participate in any attempt to cast doubt on it."  It wasn't my intent to cast doubt but I fully understood his position and continue to see him.

Before funding ASR, I was the co-founder of Whatsbestforum (WBF).  We thought by allowing both camps to state their position, life would be good.  Well, it turned out to be anything but.  The conflict eventually crept between me and my partner and I sold out my shares and got out.  I decided then to go the "pure" route and start ASR.  The name clearly states that we are committed to teachings of audio research and engineering for decades.  We don't pretend to be smart enough to invent our own rules of universe for audio and champion that to everybody else with vengeance as folks are doing here.

Net, net the response you mentioned is what you should expect if you come and make outlandish claims. You have seen me respond similarly here.  As I said, it is a jazz club and you shouldn't expect folks to take kindly to you demanding that you play country music.

You want to come and challenge our position? Do so with solid research and science driven listening tests.  Be ready to defend yourself and not cry victim with "oh they ask for controlled test and tell me about bias."  Of course we do.

Indeed, many people who have a short life in ASR mistakenly assume they are stating something new to us that we are just going to roll over and accept.  Member @kota1 for example shows up and says every cable needs to be broken in for 100 hours or the test is invalid.  We have heard these claims a million times.  Don't be the million and one member who thinks you should just throw that at us and we go, "oh, I didn't know that; thank you for that information!" 

Read the forum a bit and get educated on what and who we are.  Then participate if you need to.  You are welcome to challenge us on every topic.  Members do that to me all the time.  But be ready for heaven's sake with some back up worth more than a fortune cookie paper!

@russ69 

How is price part of audio science? 

I am not in audio science.  I am posting here.  Ask a pedantic question, you receive an answer in kind.  :)

@axo1989

Thank you for your thoughtful response. I second your final sentence, in particular.

@whipsaw

 

C’mon, Amir. As you know, Pass Labs amps have been very well received by audiophiles over decades now. They clearly have sound signatures that are pleasing to the ears of many listeners, and the suggestion that a meaningful percentage of those reactions would likely change if only those listeners were to A/B their amps with those that measure with less distortion is dubious, at best. And the same could be said of high-quality tube amplifiers.

To be clear, I don’t doubt that some listeners would arrive at conclusions that would be at odds with their long-standing, stated preferences. But given the vast weight of the feedback from audiophiles who apparently prefer amps which measure with some distortion in the audible frequencies, it is, in my view, highly improbable that their choices are primarily due to marketing-related biases.

I think ASR is—to a degree—caught in its own hermetic logic here.

One the one hand, Amir does ignore Toole’s advice that speakers should be listened to comparatively for evaluation to be meaningful. Even something as simple as setting up a curtain and turntable and enlisting helpers from his cohort of followers is dismissed. Instead he does measurements first, preps some PEQ filters based on them and on his experience, then compares "blind" original and EQ’d playback through the device under test (just one speaker). Nothing wrong with most of this (except, I would listen and take notes first—as another Kippel user, Erin of Erin’s Audio Corner does—that argument has run a few times on ASR). But I can AB between EQ and original and hear differences (most can) and that doesn’t substitute for comparative testing of (as Toole recommends) three or four speakers. Apart from the magnitude argument, it’s unclear how one type of listening is kosher and another isn’t, but that’s a longer discussion and requires more than ventilation of talking points.

But when it comes to amplifiers, ASR rarely listens, with the general justification that magnitude of differences are too small to differentiate (with some exceptions). As you can see in this thread all listening (that isn’t done by ASR with partial protocols) is routinely dismissed as sighted bias, expectation bias, focus bias etc (you too can play bias Whac-A-Mole). This despite the fact that double-blind ABX isn’t logistically straightforward. And dismissing well-defined subjective listening protocols like those published by B&K or B&O, or the expert listening we use daily to produce music. The escape clause is of course the null test, but that's also a longer conversation.

In cases where people do make a good-faith attempt to try DBT, it can also degenerate into a shambolic series of gotchas. Before my time at ASR I nonetheless read a long and winding thread where enthusiast/reviewer GoldenSound tried to respond to a challenge Amir made to compare DACs. Amir managed to misunderstand straightforward logical concepts like one vs many wrt comparison criteria, using that to reset his conditions halfway and eventually doxxed and banned his antagonist, and possibly some other posters in that conversation. I modified my appreciation of ASR ethics based on that thread. We are all flawed human beings though, so I enjoy ASR for what it offers and tolerate the weaknesses.

Dang it guys. Any electromechanical device (loudspeakers/phono cartridges) will break-in. How long does that take? Beats me. Then we have electronic components. They need to burn-in to reach their final electrical state. How long does that take? Ask the manufacturer. And finally, we have cables. Do they need to be burned-in? I don't know but if somebody says it took X hours for a new cable to sound right, I have no reason not to believe them regardless of what the mechanism might be, even if it's their brain adjusting to the new sound. So, what's the problem that needs to be disputed? 

fleschler

How do you determine the truth of claims, in your method?

Let's say a manufacturer makes claims about an expensive new digital cable.

Your ultimate method of adjudicating that claim, as far as I can see, is whether you listen and  hear a difference from another cable, or not.

So let's say Audiophile "A" listens and says "I hear an improvement in the sound" and audiophile "B" listens to the SAME set up and concludes "sorry, there is no difference in the sound."

Who is right?  They've both used precisely the same method yet arrived at contrary conclusions.

Is the audiophile who claims to hear a difference ALWAYS in the right?

Okay, how many vinyl listening Audiogoners claim that cartridges break-in or don’t?

@prof denies what he doesn’t hear or know. Yes, if you can’t hear a cartridge break-in on a good sounding system, you could be suffering with a bad short/long term sonic memory, inadequate vinyl playback system, etc. I’m at a loss here. I have never met someone who does not believe that cartridge’s break-in with use and change their sonic characteristics and usually set-up requirements. I’ve spoken to many cartridge distributors over 50 years and they all said wait until the cartridge break-in to hear what it can do.

I want to know what profession or professor of @prof is as his handle indicates?

I had never heard of ASR until I saw a post about Synergistic Research cable and snake oil. I found my prior purchase of a Foundation digital cable really bad in my system. I was offered a trial of their 3rd from the top. I tried it and was blown away at how superior it made my digital system sound. So, I followed the link and wound up with the CD trimmer comment. I made my comment and was pilloried. Further comments came with character assassinations, innuendo and just perverting my initial neutral statements concerning the trimmer, SR cable and whatever. I got mad but mentioned my recording history and work with studios. Nope, just got worse.

Despite being blocked on my Chrome IP, I easily access the site now which is why i know about current postings. I am no dummy, I started using computers full time in 1982 (I am not inferring that anyone who is not computer literate or a late comer to computers is a dummy, just that I can write programming for the software that I used in the past).

I was really mad when I began this forum. Nearly all long time Audiogon posters on this forum see what ASR is about and how angry they are at us for using our hearing ability to determine what is most pleasing in the reproduction of music. Measurements are important. Lack of measurements or less than optimal measurements does not condemn a product. Good listening determines if the result is favorable or not.

As to cartridge break-in, just admit you know nothing concerning vinyl playback.

Well, no...I won’t say I "know nothing", thank you very much. I will say I’m far from an expert and try not to make strong claims that aren't backed up with good evidence.

But, that’s me.

Am I to just fall to my knees and accept your claims as received wisdom? Or is it ok with you if I keep my critical faculties engaged?

You followed up with a bunch of claims about break in, all based on what you claim to hear. Perhaps you did hear break in. But then audiophiles believe they can hear everything under the sun - imagination and listener bias is, sorry, a fact of life and I see no reason to pretend it isn’t a variable. So, again, given I have seen many people with technical knowledge refute claims about break in, if an audiophile is going to claim break in I’ll wait for stronger evidence, in the form of measurements (or blind listening tests). You can do you, I can do me.

@tonywinga you have for all intents and purposes said you are not honest with yourself. Every day our brain presents us with ideas that are not true. Many have inflated view of aspects of themselves, while having deflated views of other aspects. Our brain lies to us all the time. But this time, this time while listening to audio, it is 100% fallible?