The Audio Science Review (ASR) approach to reviewing wines.


Imagine doing a wine review as follows - samples of wines are assessed by a reviewer who measures multiple variables including light transmission, specific gravity, residual sugar, salinity, boiling point etc.  These tests are repeated while playing test tones through the samples at different frequencies.

The results are compiled and the winner selected based on those measurements and the reviewer concludes that the other wines can't possibly be as good based on their measured results.  

At no point does the reviewer assess the bouquet of the wine nor taste it.  He relies on the science of measured results and not the decidedly unscientific subjective experience of smell and taste.

That is the ASR approach to audio - drinking Kool Aid, not wine.

toronto416

@richardbrand 
"Now I do have a couple of magnums of Henshke Hill of Grace in my basement ."
Now we are talking. Mount Mary is another wine I prefer.

@samureyex 
As much as I dislike the cult like behaviour of ASR, the Mola Mola Tambaqui did measure very well and is regarded as a very high quality Dac.

@samureyex 

While I wouldn’t characterize myself as “ an ASR supporter”, I do think ASR has provided valuable insight in several areas.  

1) DACs - Providing consistent measurements for every DAC they test is helpful, though as others have suggested, these are useful data points that are certainly not exhaustive with respect to how DACs actually sound.

2) Power conditioning products - As I mentioned previously, this is an area of the industry where I believe there is a fair amount of misinformation and even fraud.  So I think some of the work ASR has done here has helped prevent some audiophiles from spending money on products that may have no - or even negative - effects on the sound of their systems.

3) Network switches - Same as power conditioning products, only more.

 

@samureyex @jrareform 

LOL! ASR s faulted for evaluating equipment based on measurements (which are actually relevant to audio such as frequency response, distortion), and then you adduce measurements that are INaudible (capacitance, resistance, maybe also color?) to support your claim of differences? LOL!!! Can you tell from listening that cable has 10 µOhm/m vs 20 µOhm/m? (no idea about actual values, as they are irrelevant to audibility). 

The real question is, are those different values audible, i.e. human perceptible? ASR makes excellent case that they are not, based on measurements that are RELEVANT to audio. That is why Amir makes frequent reference to threshold of hearing to put measurements into context of human experience.

Yes, Amir uses equipment that can measure differences way below the threshold of hearing, which is good scientific practice. Make sure you can possibly show differences relevant to the question at hand above noise floor of measuring equipment. I use scanning electron microscopy to look at 1–10 µm structures, although light microscopy could theoretically resolve down to 250 nm; in practice less due to diffraction, but that's a different story.

Re burden of proof, of course, nobody HAS to do anything. This is in context of scientific hypothesis testing, and there H0 is always no difference. Without having to show anything, scientifically there is no difference. QED.

ASR provides a helpful ’yardstick’ of objective measurements. In a world where many are either forced or choose to buy components that they cannot audition, I believe ASR provides a valuable service. And even for those of us who have had a chance to audition prospective purchases, when caught betwixt and between two pieces of gear based on our subjective evaluations, again, ASR provides a helpful yardstick. Cheers!

What are the possible aspects of music you can hear?
Frequency
Amplitude
Duration
Accuracy to original signal

These are exceptionally easy to test, the flavor of wine is impossible to test we don't even know exactly how taste or smell actually work. 
Audio is simple to test and duration is the part of physics that mankind is the most adept at, musicality, fluffy descriptions of ambiguous terms only cost you money and show how gullible we are.


 

@donavabdear 

i’ve never thought about it in that way and that’s an interesting perspective. Not sure I completely agree with the conclusion, but certainly an interesting premise

@mdalton Providing consistent measurement for DACS, but the data doesn't seem useful. I or you can't look at the data and say this better measured one will sound better. Which is where I question its usefulness. If I can't predict the sound quality based on the measurement, then what good is the measurement?

@oberoniaomnia You mention a point where I just want to bang my head against a refrigerator every time I hear it. Something ASR loves to use as their defense also.

This is one of ASR biggest problem.

"If it measures the same, it sounds the same".

"if it measures different, it’s beyond human hearing, so it also sounds the same".

Like come on give it a break.

1) Cables sound different and you guys measure it to be the same.

2) Told you guys to measure capacitance and inductance of a cable and you say it’s not audible.

3) ASR says they are calling out the BS in the industry but they have become the same demons they repelled. ASR promotes a bunch of mediocre sounding Topping products as world-class.

4) We have good sounding products and you measure it bad

5) You guys measure bad sounding products and say it’s amazing.

Hear me out, what if a poorly measured product is actually a great technical product and hence it sounds good? That’d actually make more sense.

Erin showcased a significant problem with himself and with ASR. For those that don't know, Erin is a part of ASR. He's also purely a measurement guy, with a bit of subjective listening in his reviews.

Here's what happened. There's a Mcintosh amp and a pair of Mono class D amplifier that are db-tuned. Cannot be more different on measurements. Virtually apples and oranges. And the issue? Erin cannot tell the difference. Not a single difference. 

What is the point of all these data when the end user can't tell the difference? You might say, well it's just Erin, but I've come across many people that can't tell the difference. People at AVS. 

Respect to Erin, at least he doesn't try to hide the truth and says it like it is. HUGE RESPECT. If Amir did the same test he would fail just as miserably. 

@laoman Interesting product. Thanks for bringing it up. Bruno is a measurement-first engineer. Anything he makes is guarantee to measure well, but is not a guarantee they will sound good.

There is so much discrepancy between how the Tambaqui sounds, vs its pricetag, vs its measurement. I’m gonna make an argument and say it’d have made more sense for everyone, Amir included, if the Tambaqui measured bad instead.

Full disclosure: I have not heard the Tambaqui

1) Based on owners impression and reviews. The Tambaqui performs on the level of Chord Dave, and DCS Bartok. The pricetag reflects their performance as well. Chord Dave - $14,400. Tambaqui - $13,400. Bartok $20,950. Bartok is 50% more but I digress (I've been told Bartok used to be very close in price to the Tambaqui).

2) Bartok measures BAD. Dave measures BAD. Tambaqui measures GOOD. Huh?

3) Topping D90se - $900. Measures GOOD. It measures so good that it is nearly identical to the Tambaqui. $900 vs $13,400. Nearly the same measurements. Huh?

4) I’ve owned the D90se. It sounded bad, subjectively bad. There is just no way the D90se would sound as good as the Tambaqui despite the measurements. The measurements for these 2 products make no sense, no sense in price, no sense in performance.

So to conclude, the measurements make no sense, Amir once again proves his data is meaningless. 3 products of similar performance, 2 measured poorly, 1 measured great. Makes no sense. Logical conclusions cannot be found at ASR nor from Amir. The only thing that made sense here is the pricetag (kind of).

 

This is what Amir had to say at the end of his Tambaqui review,

"Since I am not the one paying for it for you to purchase it, it is not my issue to worry about the cost. As such, I am happy to recommend the Mola Mola Tambaqui DAC based on its measured performance and functionality."

He’s happy to recommend the Tambaqui when the D90se can be had for $900. Such a humorous clown. 

Laugh out loud, roll on floor laughing again until I poop my pants. 

Do not buy anything based on ASR recommendations.    Buy based on whether you like the sound.  And there are plenty of products they bash soley on specs that actually sound excellent.  So don't pass on a product just because they trash it.  .   That's my take away.    

 

 

This post has acquired an interesting and persistent energy. ASR is clearly neither trivial nor unpersuasive or why would so many try to denounce it?

I occasionally go to ASR to read their reviews. Their contributors don't appear to be particularly reductionist or dogmatic. If you know what they are using for testing, you can take that as a data point and move on. Their peanut gallery in their comment sections are what you will find anywhere, people who opine based on the review and not anything more.

What no site appears to do is true blind listening tests using a standard setup for 2 channel audio, and using self-validating methods (e.g. testing the same system twice to look for variation of the listener's attention and judgment.) 

@samureyex 

Well, I notice that you only responded to one of my 3 examples, lol!  But with respect to DACs,  I think each of the measurements is intended to reflect some specific aspect of sound.  Whether you like that quality is of course up to you.  But the measurements are intended to, and I think generally do, correlate with a specific quality.  For example, a high signal-to-noise ratio means the signal produced by the DAC is high compared to any background noise or interference, which in turn means that the DAC is likely to sound “clean” and retrieve a high level of detail.  So the benefit of ASR measurements, which was one of your questions, is that they help informed consumers of the data know which DACs they might prefer if, for example, detail retrieval is one of their listening priorities.  

Finally, I’ll reiterate that I don’t believe that these measurements tell the whole story about any DAC.  Two of my four current DACs measure among the best ever measured by ASR.  But my favorite of my 4, while it hasn’t been measured by ASR, would almost certainly not measure as well.  And it may well be that my fave is a little less detailed, or has a little more second harmonic distortion, or maybe it just has a little more bloom in the midrange cuz it has a better power supply.  I’m not sure, and I’m totally fine with that.  I think some of us have a tendency to get way too dogmatic in this hobby.  I include some of those at ASR as well as some here at  the ‘Gon.  Bottom line, I believe ASR has added a lot to the knowledge base for many in our hobby, and I appreciate that.

@samureyex 

What has the better predictive value for the sound of a DAC: SINAD or price? I go with SINAD. Not perfect, but better. Scientifically expressed as higher factor loading in a PCA.

Why are expensive DACs generally reported to sound better? Because buyer would otherwise have to admit of having wasted money. Pure psychology.

I just bought a Topping D50III ($250) based on ASR assessment. If memory serves me well, it measured SINAD 123 vs. Molamola at 124. So the Topping appears to be good bang for buck, particularly for my little secondary desktop system. As an aside, the D50III gets some recommendations by audiophiles. I compared it to my HoloAudio May L2 ($5K; not in ASR listing) in my main system. Plugged both into Roon server, volume matched, linked the two sources in Roon, plugged both into preamp, so can switch in an instant. Listened to a bunch of different tracks and music styles from EBM (attack/punch) to indie pop (soundstage) to chamber (decay). On some I liked the May better, on some the D50III. Essentially, overly analytical recordings sounded better on the more laid back May, whereas more relaxed recordings sounded better on the D50III. The differences were very subtle, much less than different tubes, headphones, or cartridges. Bad recordings were equally unmasked by both. So a $250 DAC is not clearly worse than a $5K DAC. Price difference: 20x. Shockingly (not) it depends a lot on the recording.

ASR serves as a good buyers guide for those of us who rather listen to music than to gear. Not the only one, but a good source.

Did I waste $5K on the May? Arguably yes, though the D50III did not exist when I bought the May. Am I replacing the May with another D50III? No. I already have the May and it looks nice, but admittedly does not sound clearly better. I still have to compare the May to the D50 [OG] with SINAD of ~90

Re cables, again, you do not hear capacitance or resistance or inductance, just as lille as you hear color. As pointed out by others here, go start a double blind testing series with listeners, and report your findings. If the EFFECT of cap/ind/rest can be heard in sound waves, that will be most fascinating. A possible EFFECT of cap/ind/rest is frequency response, noise etc. and that is precisely what ASR measures.

@mdalton I only responded to 1 of your 3 examples because I am very unfamiliar with the other 2.

All of the 3 mentioned dacs are exceptionally clean and have exceptional detail retrievals. Which makes no sense because Amir measured the Dave to be very poorly.

@mdalton I sense a bit of disappointment in you in that I only responded to the dac section. I’ll talk about the power conditioner part here. I’ll do you a good one.

The Puritan 156. A HIGHLY beloved product among audiophiles. How does Amir measure it? Well let’s quote Amir himself.

Conclusions
As you see, I have run a number of tests to give the PSM156 ample opportunity to show it can do something to improve audio but it can’t even move the dial one hair. There is no indication or logic that would tell us that it can make an audible improvement. Yet the company says this in the feature list of this product:

The only thing reliable about ASR is severe inconsistencies, which is exactly what good measurements avoid.

Calling ASR a cult and selling snakeoil is out of good conscience, a very accurate description.

 

@samureyex 

Think you may have intended your post referencing the Dave for someone else.  I don’t have one.

Regarding your post about the Puritan, you left out what mattered in Amir’s conclusions, which contained no inconsistencies.   Here’s the full conclusion: 

“Conclusions
As you see, I have run a number of tests to give the PSM156 ample opportunity to show it can do something to improve audio but it can't even move the dial one hair. There is no indication or logic that would tell us that it can make an audible improvement. Yet the company says this in the feature list of this product:
 

1630123343700.png



None of these things is observed in the measurements. 

The device however seems to have real filtering in there as opposed to toy implementation we see in other audio devices. So if you have audible mains related interference that is above a few hundred Hertz, then the PSM156 may have an effect.

As an audio fidelity improvement device, I can't recommend the Puritan Audio PSM156.  As a mains filter in general, it seems to be well built and does a far better job than typical consumer gear sold for this purpose.”

@oberoniaomnia

If you can not hear the difference between the Topping and the Holo May, then I am pleased for you as you do not need to spend more money to get far better sound.
I, on the other hand, think Topping sounds really harsh in the mid level, particularly with female voices. I found it grating after 20 minutes. As I listen to a lot of classical music as opposed to doof doof, there was no way that I would ever buy a Topping.
I am glad yours is still operating. You are aware of the poor quality control that even many ASR members complain about?

@laoman His (oberoniaomnia) post just proves ASR measurements make no sense. 99% of R2R Dacs measure terrible vs the Delta-Sigma Topping dacs.

For him to even say the Topping and Holo May are more or less equal is just wild. Because the measurements wouldn’t support his statement. Keep in mind how good his Topping D50 v.3 measures. It should on paper, poop all over the Holo May.

I’ve previously stated the Topping D90se (previous Topping flagship and a measurement beast) is subjectively bad.

1) Small soundstage

2) Very poor actual dynamic range. Let’s say you have 10 different sound in the Loud range, 1-10. The Topping can only do 5-6-7 for all 10. The same for problem for soft notes.

3) Poor spaciousness. Lack of air and space between instruments. The small soundstage makes it worse.

4) A very homogenized and boring sound. Because of #1 #2 and #3, Most instruments and notes have similar loudness, It just sounds very similar tracks after tracks. At the end of the listening session It would feel like 1 very long track.

5) Flat dimensionality. Clarity is high but transparency is low. Everything is upfront,

Compared it to my old Anedio D2 dac ($1,200) at the time. The Anedio is leaps better in every aspect, except vocal’s warmth. If I didn’t pay for the Topping myself I would’ve thought it was a $300 dac. All of these are subjectively bad traits in audio. Congrats to D90se for being a measurement beast and sounding the way it does.

I don’t see the purpose of ASR existing anymore. If you believe in ASR’s measurements, You should only buy Topping products. Get their dacs, amps, and preamps.

There is no point in measuring X products. Just get a Topping and be done. Which Toppings don’t matter either. 123 sinad is not going to make a difference vs 124 sinad.

Buy Toppings, Buy Benchmark, Buy anything that Bruno puts out. These 3 are going to put out SOTA measurement products.

@mdalton I was intrigued by your exerts from the PSM156 review in ASR so I read it. My suspicion was confirmed, Amir tested the device with one piece of hifi gear attached at a time. My understanding is that higher end power conditioners have several functions: to supply non limiting current to a number of devices simultaneously, to filter noise that may be in the AC supply line, either coming through the mains at the breaker or imparted to the line between the breaker and the outlet, reduce noise in AC lines emanating from the attached gear or power cables, and potentially surge protection. In Amir’s test, he looked at the performance of the Puratin device with respect to noise in the AC supply line and looked at the measured performance of one piece of attached gear.

Hifi systems in practice can be quite complicated, with multiple devices connected to a power distributor/conditioner. For example, my system is a hybrid 5.1/2.1 home theater and two channel system with four different sources, both digital and analog, a large receiver and a subwoofer. Power from the 20 amp breaker is supplied by a single run of 10 gauge romex to a medical grade outlet. I have a stacked rack arrangement with seven different power cables and fourteen different lower level digital and analog cables all in rather close proximity. Most of my power supply cables are upgraded except for an old Blu-ray player and a vintage turntable which have attached “lamp cord”. The receiver is plugged directly into the wall and everything else is plugged into a power conditioner/distributor. I suggest the opportunity for some electrical noise generated in digital devices or power supplies to affect other devices or cables nearby is significant.

My system definitely benefitted by replacing a sturdy but simple non-surge protected power strip with a more substantial power conditioner/distributor, and by again by replacing the supply cable from the wall to the conditioner/distributor with a better cable (the single biggest cable improvement I have made in over 20 years of tinkering). I do not know if this benefit of adding a new conditioner/distributor and supply cable had to do with less current limitation due to higher gauge internal and external wiring, cleaning up AC from the mains, or reducing the impact of noise generated in the attached equipment.

What I do find challenging is a review of a power conditioner designed in part to tame electrical noise in a system of attached components by connecting and assessing one device at a time. The review of the Puritan Audio PSM156, perhaps more than any other on ASR, points to the weakness of the reductionist approach for evaluating products that were designed to reduce noise and improve the sound of a complex system by looking at one variable in isolation. To strain the oft used audio-car analogy, this is like saying “I tested the Corvette on the skidpad and while it did what the manufacturer claims I cannot recommend it for driving in traffic”. This also reinforces the need to test equipment in your system in your room with your ears. No two circumstances are alike, and as always, YMMV.

kn

@knownothing 

Amir’s conclusion actually says “So if you have audible mains related interference that is above a few hundred Hertz, then the PSM156 may have an effect”, which is decidedly less dogmatic than my friend @samureyex.  If you read Amir’s stuff on other power conditioning products, what he pretty consistently argues is that AC filtering is usually unnecessary because well-engineered products always have, and must have, power filtering built in that occurs before the conversion of AC to DC.  

My take is that in your use case, you are arguing that the interaction of multiple devices has generated noise that has been dumped back into the AC mains, and that one or more of your devices has failed to filter that noise before conversion to DC.  And this would be very easy to measure.  Now Amir, who can get dogmatic himself at times, might argue that any gear that fails to filter out that noise is poorly engineered.  I have a different view.  

I have alot of tube gear, which is far more susceptible to noise than solid state, particularly when you’ve got the low level signal involved with an analog front end.  I’ve had annoying noise gremlins that required a fair amount of experimenting to eliminate.  In one case, I ended up actually having to remove a filtering device that was somehow adding noise, and replacing it with a simple star-grounded power strip.  During my experiments, I came up with a crude but effective way of measuring my results: I used a db sound meter app on my phone and held the phone next to the speaker while turning up volume to 12:00.  My control sample was my digital front end, which even with all my tubes was very quiet.  I then would do the same when I engaged my tube phono pre into the system, trying all my different power combinations.  As I said, crude, but highly effective (for me anyway).  YMMV.

One thing everyone can agree on about ASR is that they should really ditch the panther. Odd that they haven't been sued.

@devinplombier    I agree, Owens Corning has to pay to use the panther. You think Amir is paying to use the panther, I doubt it.

You think Amir is paying to use the panther, I doubt it.

Sure he paid, he bought it.  Its his panther.

@mdalton +1

I suggest it is not just gear that is susceptible to effects of noise or responsible for generating noise, but also interactions between low and high current cables (AND THEIR CONNECTORS) that are operating in close proximity to each other and may create distortion, slight changes in timing or actual signal loss.  Until everything is hooked up together, you really don’t know what you got.

kn

hilde45 Unless you are weight training which can make using a scale irrelevant and possibly detrimental to weight loss.

The subject of ground loops is IMO under-appreciated in the audiophile community.

There's nothing mysterious going on, it's just physics. But it can be challenging if not confusing to learn and apply. Sorting out fiction from fact may be an issue for the layperson as well.

Bill Whitlock has put out some of the most accurate and digestible material on this topic. Search "Bill Whitlock ground loops" and you'll find at least a few freely available papers and powerpoint decks he has authored.

It's well worth having at least a basic knowledge of safety grounding ("earthing") and how it relates to noise in audio systems. First and foremost to keep yourself and others safe with a proper installation, but also to address noise issues effectively.

Have a look on the Jensen Transformers website for application notes AN007 and AN009 for example.

Have a look also for "indy-aes-2012-seminar-w-notes-v1-0.pdf"

I believe Amir started with good intentions to provide useful information. The emergence of a vocal, sometimes arrogant and nasty, ASR following turns many off.  Amir has done very little to rein in the most offensive ASR faithful. 

Amir himself automated a search process for mention of ASR in other forums so he could jump in to defend himself.  That defense typically involved him arguing everyone else is wrong, he is right, ad nauseum in a long and repetitive thread.  He defends that approach as justified due to so many attacks leveled against himself.  That is ironic considering how many times Amir and the ASR faithful have attacked other audio businesses as dishonest know nothings.

Amir promotes his technical knowledge as expansive and extensive.  Yet he asked Audio Precision to make their analyzer more "fool proof".  Amir claims vast technical knowledge, but is unable to reliably operate the AP analyzer.

Amir promotes Topping and other economical products as measuring well so they must sound good.  That is good information if it stopped there.  Often, Amir and the faithful insult others that choose to spend more as fools.  Ironic others are fools for spending as they choose while Amir owns personal audio equipment well into the 6 figure territory.  

Amir reveals his other business venture in order to claim immunization from any conflict of interest or bias.  He and ASR faithful frequently point to that transparency as inviolate fairness and independence.  Regardless, Amir is no more pure as the driven snow as any other reviewer.  

Finally, Amir would be better served to present is measurements and dispense with the chest thumping I am right, I am right, listen to me because I am right mantra.

 

 

It’s a free country.  It is a shame but Offensive is very in these days all over.  Who knows why?  Just ignore the offensive and focus on the good. No sense getting all uptight over it.  

@texbychoice

Now that’s a load of unsubstantiated BS - heavy on innuendo and MIA on facts.

Mostly you sound angry and butthurt. It’s almost as if Amir had disparaged one of your products or something

Amir himself automated a search process for mention of ASR in other forums so he could jump in to defend himself.

Amir had been conspicuously absent from this thread, so his nefarious "search process" doesn’t work very well.

... Unless he is trolling this forum under a fake identity. But honestly what kind of creep would do that?

 

One of the topics that always comes up when objectivists from ASR and elsewhere are challenged is to present the alternative to machine measurement being blind or DBX listening tests.  The conversation usually devolves into a recitation of 1) the general failure to back up claims of “sounds better” with procedurally and statistically valid proof, and 2) when such studies have been attempted and published they are generally inconclusive or fail to refute the null hypothesis that there is no difference between pieces of equipment A and B.  Over time, these “tests” have been done with human subjects on electronics and cables, with an often cited study conducted in the 1980s by a marketing professional testing perceived differences in the sound of various amplifiers across a range of prices.

I have been perplexed by “findings” that routinely show no perceived difference between different gear or wires when limited blind tests I have done with friends show clear differences (not always tracking linearly with price).  When I report the findings of my admittedly amateur analysis as a counterpoint to the objectivists, my methods are invariably questioned.  The consensus feedback is “you must be wrong, you made a mistake, or you are lying”.

So what is going on here?  Why do so many in this hobby insist they hear subtle and not so subtle differences when they swap a cable or a DAC?  Especially when objectivists like Amir can’t measure a difference THAT SHOULD MATTER, and what blind testing that has been done is either inconclusive or unsupportive of a real performance difference.  I have some ideas and they generally fall into these categories: 1) some people have better hearing or listening skills (this is where the original wine tasting analogy in this thread makes a comeback), 2) the gear, room and/or listening position are not optimized to allow for a difference to be heard, and 3) the experimental or sampling design is flawed and has not been optimized to detect subtle differences.

I will explain.  The portion of the world population that are “audiophiles” is small.  And just because a person likes music and can afford to lavish time and money on the hobby does not guarantee they have strong listening skills - and if they love going to rock concerts or even spent a lifetime on a classical concert stage or in practice rooms, their hearing may be compromised.  In the general public, there is a gradient of hearing acuity but generally poor training in listening to reproduced music, so who is included in such a test matters.

Choice of gear matters.  What is the synergy between the components selected?  Are they selected in such a way to accentuate the differences in the piece of equipment being assessed?  Is the room well designed and acoustically neutral?  is the power supply from the wall clean and adequate?  Does the system have power conditioning?  Is the seating position optimized to allow for maximum resolution by the test subject?  Testing a group of people in a room at the same time where only one person is located in the sweet spot of the speakers would not be the ideal way to test the soundstage reproduction characteristics of a DAC or cable.  Using headphones for the test could reduce this variable, but headphones are generally a poor substitute for well-set up and sourced speakers in good room for testing reproduction of soundstage.

Finally, what is the test regime for the subjects.  Are they allowed adequate time to acclimate to the sound characteristics of the system and room before making a change for the test?  Most audiophiles spend multiple days analyzing the sound of a new component and cable, swapping them in and out before deciding if there is a difference or an improvement.

If I was tasked with developing listening tests for high end audio gear, I would screen the listeners to determine both their hearing acuity and their listening skills.  The target audience for these products are not teenagers listening to poorly produced streamed music with their smart phone and ear buds on the school bus.  I would select the system, the room and the seating position to optimize any inherent differences in the items to be tested.  I would probably throw in some headphone listening as well to remove many of those variables, as there are elements of reproduction that headphones excel at.  And I would partner with an expert in human subject tests, and optimize the testing regime to maximize the likelihood of a statistically valid test of the null hypothesis that there is no difference between item A and item B or no change (X) with adequate controls.

That is a lot of work to determine with some statistical rigor if a Mola Mola DAC sounds better than, the same as, different from, or worse than a Topping DAC.  I have done blind wine tastings as part of a wine club at one time in my life.  I found that I was not the most adept at detecting differences in the different wines and when I could, I did not have the vocabulary to describe what I was tasting.  It was enough for me to recognize that I liked the stuff in bag number #3 and compete with others to drain that first.  The most pompous member of our group had gone to Harvard from kindergarten through PhD, and thought he had an excellent pallet.  He did not.  His very humble wife however did have an excellent pallet, and routinely identified differences in the wine and the associated characteristics on a tasting wheel.  

In wine tasting as in audio listening, some people “have it” and some people do not, but many people can enjoy drinking or listening in their own way.  As the OP noted, engineering and food science can be applied to assure a certain level of quality, but there are very many other variables that go into how we experience and enjoy wine and hifi.  I find in moderation they are best enjoyed together.

kn

You make excellent points about the rigor that must be applied to testing as well as other potential issues. I suspect most of us are at an age where there are not a lot of golden ears in the group.

I'm most certainly not an ASR follower. I've invested in relatively high end cables, a good DAC, etc., and I generally believe that electronic components matter a great deal. I don't, however, dismiss Amir, Gene, Erin and other objectivists out of hand nor do I have an emotional response to their point of view because it differs from mine.

I'm a little closer to agnostic particularly when it comes to some of the cables, switches, etc. What I know about science wouldn't fill a thimble compared to some of the EEs on here but I know a little about business. I know that if a cable company-pick any of them-set up a juried, independently verified completely professional blind testing that was seen as unbiased with no room for tricks-ABX-whatever-and filmed it live and if the participants in statistically significant majorities picked their cables over competitors, or even much cheaper cables, switches, etc, that company would experience a great surge in orders and would reap a big reward. These companies are run by rational business leaders. It bothers me that as far as I know, none of them do that, because why wouldn't they? I'm not an "objectivist" but that bothers me.

Not looking for a fight-I'm happy for folks who find products that enhance their experience even if its buying rocks to put around the room-whatever works for you. Its an interesting question though.

Our audio group routinely does simple double blind tests and I can tell you it is not a relaxing experience. I am convinced that people listen very differently under testing conditions than they do normally. 

But the real issue is why should you even care? After decades of 2 channel listening, I no longer doubt my senses. To me this is ridiculous argument that will yield nothing. A discussion which has been going on for decades, I just dont remember there ever existing a worse ambassador for the measurement side than Amir.

@kerrybh 

I have similar views.  Product types for which I have a healthy amount of skepticism include power conditioning, high-cost streamers, and network switches (omg!).  Manufacturer claims regarding their low noise levels, jitter immunity, etc. could be easily documented, but in virtually every case they don’t.  Hmmmm….

@devinplombier 

Now that’s a load of unsubstantiated BS - heavy on innuendo and MIA on facts.

All facts. Read many of the threads on ASR.  Name calling and personal attacks routine behavior.  Your comments follow the ASR model.

Amir jumped a thread here and went on and on and on posting numerous charts, graphs, and long verbiage. Amir himself mentioned the search algorithm.  Fact.

There is a video of Amir asking AP engineer to make the analyzer more "fool proof".  Fact.

Amir himself documents his personal system and talks about cost.  Fact.

Transparency about Amir's other business comes up in defense of his claimed absolute and unquestionable impartiality.  Fact.

Mostly you sound angry and butthurt.

You are hostile and butthurt by facts, not I.

It’s almost as if Amir had disparaged one of your products or something

I make no products of any kind.  You reveal yet another ASR minion tactic of leaping to a ridiculous assumption.

 

Hahaha links to all these facts please?

Amir jumped a thread here and went on and on and on posting numerous charts, graphs, and long verbiage. Amir himself mentioned the search algorithm.  Fact.

There is a video of Amir asking AP engineer to make the analyzer more "fool proof".  Fact.

Amir himself documents his personal system and talks about cost.  Fact.

Transparency about Amir's other business comes up in defense of his claimed absolute and unquestionable impartiality.  Fact.

 

 

@devinplombier

Do something besides spewing nonsense.  Put forth a little effort and do the research yourself.

Amir is absent from this thread because of the damage he did to his "brand" the last time he got involved.

Textbychoice is correct with numbers 1,3 and 4 listed above. Didnt know about the engineer video.

 

Calling an audio marketing ideology "science" reflect a tendency of our  dominated social fabric...

Arguing about the sound quality of our gear at least make an appeal to our hearing  sense, educated or not, trained or not...

 

Acoustics rules which include psycho-acoustics...

my ears rules in my designed room...

 

This thread cannot die.  It has been so utterly entertaining

@samureyex @knownothing thanks for posting my exact thoughts.  Making it super easy for me over here laugh

ASR, I think by design, takes a fairly extreme and militant stance precisely to get responses. Not a question of being pertinent rather one of being hard to ignore.

 

Right!

Polarising people in two opposed groups is the best way to win an audience... In many aspects of life...

 

ASR, I think by design, takes a fairly extreme and militant stance precisely to get responses. Not a question of being pertinent rather one of being hard to ignore.

 

 

Measurements ≠ Subjectively Good Sound

I suspect that those who believe measurements are everything are deceptively bias against costly “unaffordable” components. For some, it may be emotionally easier to digest a lie and talk oneself into believing the lie than to face the unpleasant truth of unaffordability of desired components.