The Audio Science Review (ASR) approach to reviewing wines.


Imagine doing a wine review as follows - samples of wines are assessed by a reviewer who measures multiple variables including light transmission, specific gravity, residual sugar, salinity, boiling point etc.  These tests are repeated while playing test tones through the samples at different frequencies.

The results are compiled and the winner selected based on those measurements and the reviewer concludes that the other wines can't possibly be as good based on their measured results.  

At no point does the reviewer assess the bouquet of the wine nor taste it.  He relies on the science of measured results and not the decidedly unscientific subjective experience of smell and taste.

That is the ASR approach to audio - drinking Kool Aid, not wine.

toronto416

Showing 32 responses by samureyex

@prof Your argument relies solely on this simple equation

Good measurements = good sound.

Great measurements = great sound.

Bad measurement =  Bad sound. 

Different wording but it all implies the same thing. That if something measures good, it must sound good. Which is completely untrue and many can attest to this, Both consumers and designers. I have seen respectable designers come out and say they purposely use a worse measurement component because it sounded better than the better measured one.

If measurement was a good indication on the sound quality. ASR should only measure products and not do any listening. The Topping D90SE was (is?) the best measured DAC ever, and how does it sound? Honestly not very good. 

Going back to the measurements. There are products that measure just god awful and they sound great. It doesn't take much to realize there's something amiss in the way ASR do their measurements. 

@hilde45

"Another ASR hating thread. One need not say that listening does not matter in order to see the value in measurements.

Are there ASR folks who think that only measurements matter? Sure. But that does not mean that the measurements done cannot be helpful at all. (I mean, I’m not a fanatic about my weight on a scale, but it helps to know when I’ve gained 10 pounds.)

I honestly don’t understand why ASR is like Voldemort to some folks here."

 

What you said immediately clicked the bulb in me. IMO, the error of ASR resonates a lot with the error of your analogy. About gaining 10 lbs.

You could be gaining 10 lbs in water weight, Or 10 lbs in muscle. Or 10 lbs in fat, or any combination of these things. You can look quite different depending on how you gained your weight.

You could weigh 210 lbs and be visibly/noticeably thinner than when you were 200. How is this possible? Well you could’ve gained extra muscle mass, lost fat in the process, and because muscle weights A LOT more than fat, you will look slimmer even though you are heavier.

Kudos to you for making a perfect analogy to weight gain on a scale vs what ASR does.

@analog_aficionado 

Thanks for sharing your invaluable knowledge. A quick word and disagreement on Amir. I don't think Amir means well. I've seen him spoken out so many times. His demeaner is "my way is the right way". Never a good trait for any scientist. 

What you said about SINAD reminds me of the TV industry these days. It's all about the nits and how bright the TV can get. Completely forgotten the many other important aspects that make a TV good. 

 

Here's how you know ASR is borderline useless. There'd be a new product, and the ASR people would dissect and read all the data, and then something magical happen.

They all wait for subjective user experience.

If your scientific measurements are not reliable, then what are we even doing here?

Point at a random well measured speakers, the fact that no one can confidently say it will undeniably sound good by just looking at the data alone, that is troublesome.

The fact that ASR has rankings for speakers is troublesome.

Some speakers are better at certain things. Bass, mids, vocals, treble clarity, separation, imaging, 3d holography. 

Just shows these people only listen at numbers and graphs. It's fine, if you get pleasures from listening to graphs. The world still spins.

Guys, I think we need to all take a step back and realize 1 EXTREMELY IMPORTANT DISTINCTION.

ASR DOES NOT represent nor speak for all of the audio science community.

ASR = ASR

ASR ≠ Measurement crowd

An extremely important distinction. There are many people, both enthusiasts, and electrical engineers that do not align with how ASR do things and how ASR conclude data. 

You can still agree with science and measurement and still disagree with ASR. Sometimes we miss the too obvious.

I say what I said above because I see too many people equate

ASR = Data & Science & Facts.

Sinad and THD do not represent the entirety of science nor audio.

ASR most praised products do not sound good.

I can tell you exactly why the Topping D90SE is a mediocre product. Clearly distinguishable in a blind test.

Keep in mind the D90SE is the best measurement DAC of all time. There’s a clear disconnect here.

While on the topic of the best measured dac ever. Benchmark designer made the Benchmark Dac 1 and for years he insisted it's the best dac. It cannot be improved further.

But people did not like the Dac 1. Some years later, the Dac 2 was released.

If you ship a non-traditional item to ASR for review, no matter how good. Amir will absolutely destroy that product in his measurements. This I can guarantee you.

He absolutely obliterated, massacred the Chord Dave he reviewed. There is no shortage of great and beloved products that he had destroyed with his reviews. But strangely no products he endorsed are beloved.

I see the same mistakes being made again.

Being against ASR is not the equivalent of being against measurement. Hold up a dac in your hand. It is a product of pure science, and engineer. To say that we don't trust science or data is pure nonsense.

I said it in one of my earlier post, everyone at ASR wants data and measurement but then something magical happen, they wait for subjective reviews and user experience.

Let's go back to the extreme basic. In Amir's testings, all cables proved to be the exact same. To be as frank as I possibly can, anyone with a decent system and 2 functioning ears would know this to be false. CABLES DO MATTER and if your test says otherwise, your test is wrong.

If you can't do a cable measurement. What exactly can you reliably measure?

@oberoniaomnia

There are thousands of people like you, Amir & Gene & Erin included, that cannot hear a difference with cables. Doesn’t mean there isn’t. Different metals of cables, purity, capacitance, inductance, quality of connectors, length, geometry, type of shielding, thickness. All of this come together to form a cable. To say cables have no difference is saying all of these aspects don’t matter. Which if you give it 2 minutes to think about, is quite ludicrous. Especially for a "no-nonsense scientist" such as yourself and Amir.

Erin has stated he cannot hear a difference between amplifiers, does not mean there’s no difference.

Leave the idea of cables aside for a minute. I’ve encountered hundreds, and perhaps even thousands of people that have said they cannot hear a difference between DAC, AMP, and pre-amp. These people, like you, truly believe there’s no difference with certain components.

The facts remain. If you cannot measure a cable, what can you really measure?

@decooney 

"It only matters to those who can hear a difference. "

 

This is actually quite profound.

A brain exercise for those who think all cables sound the same because ASR measured them to be the same:

The terminal connector of a cable cannot be pure copper, because copper cannot hold a shape well enough. It's usually a mix different metals along with copper. This will degrade the signal. 

The actual brain exercise, If you have a cable that degrades the signal, some will degrade less, some will degrade more. Assume you have 1 cable of each, you think they would sound the same?

This is just the terminal connector we're talking about, there are a lot more aspects that haven't come into the picture. 

@oberoniaomnia Regarding cables, all you need to know is the capacitance, the inductance, the purity, and the quality of the terminal connectors, and the method in which the wire is meld with the connector. You think these things, on an individual level and as a whole don’t matter to the sound? That’s a tough pill to swallow.

Silver is 6% faster than copper, you think that doesn’t change the sound?

@devinplombier I have evidence, straight from Erin’s mouth that clearly stated he cannot hear a difference between any neutral sounding amplifier. Imbecile or not. I am not surprised. At least Erin has the balls to admit it.

https://www.youtube.com/watch?v=d5KfafDpXvQ&t=5s

It is in this video. Forgive me I forgot the exact timestamp. He stated the only reason he heard a difference between said tube amp vs his neutral amplifier was because the tube amp was not neutral. He also stated he cannot hear a difference between 2 neutral amplifiers. As we all know, an amplifier contributes to the sound much more than how neutral or not neutral it is. And if said reviewer only relies on the neutrality to spot a difference, well that's bad. But at least he admits his limit which I can respect.

@richardbrand Are you telling me silver and copper sound the same? They have different electrical properties, different capacitance and inductance, so the question is, do you think they both sound the same?

Also pure copper is far from an "excellent connector". For the same reason you stated, its softness and malleability, the opposite of what a good connector represent.

https://www.youtube.com/watch?v=GMjBUvEHVdc

Here’s another video of Erin with even more amplifiers. A traditional Mcintosh with a house-sound and piss-poor measurement, vs a pair of Mono class D with "better" measurements with a focus on pure neutrality.

What’s the listening result? He cannot hear a difference. Only past 95 db where he starts to feel the Mctinosh has an edge in power.

Makes you wonder why even bother with Sinads and THD in the first place. Between you and me, I’m willing to bet Amir can’t tell the difference between a tweeter and a bird.

@richardbrand Ain't no way the binding post will even up your bumps. Even if it did, you still wouldn't want a pure copper spade, the oxygen and humidity will destroy it over time.

So this circles us back to the initial point. A good connector would have a mix of different metals, which will inevitably cause different degree of signal degradation. How much will depend on the technology and alloy involved. 

Now you have 2 cables of varying signal degradation, in which universe would you expect them to measure and sound the same?

@devinplombier "Pure copper makes an excellent spade connector"

A pure copper spade will have uneven bumps resulting in poor connection.

@oberoniaomnia You want burden of proof? Ok I will give you burden of proof.

Go measure the capacitance and inductance of cables, they can vary greatly.

Low inductance has a sound profile.

High inductance has a sound profile.

Low/high capacitance each has a different sound.

Of course there are many other things that also affect the sound of a cable, but this is enough for the burden of proof.

@jrareform 

Well said. 

I'd like to ask the ASR supporters.

1) What has ASR actually done to propel this industry forward?

2) Name a product(s) that has world class measurement from ASR that also happens to be beloved by the experienced audio community.

3) Name a product you absolutely love thanks to ASR recommendations.

Erin showcased a significant problem with himself and with ASR. For those that don't know, Erin is a part of ASR. He's also purely a measurement guy, with a bit of subjective listening in his reviews.

Here's what happened. There's a Mcintosh amp and a pair of Mono class D amplifier that are db-tuned. Cannot be more different on measurements. Virtually apples and oranges. And the issue? Erin cannot tell the difference. Not a single difference. 

What is the point of all these data when the end user can't tell the difference? You might say, well it's just Erin, but I've come across many people that can't tell the difference. People at AVS. 

Respect to Erin, at least he doesn't try to hide the truth and says it like it is. HUGE RESPECT. If Amir did the same test he would fail just as miserably. 

@oberoniaomnia You mention a point where I just want to bang my head against a refrigerator every time I hear it. Something ASR loves to use as their defense also.

This is one of ASR biggest problem.

"If it measures the same, it sounds the same".

"if it measures different, it’s beyond human hearing, so it also sounds the same".

Like come on give it a break.

1) Cables sound different and you guys measure it to be the same.

2) Told you guys to measure capacitance and inductance of a cable and you say it’s not audible.

3) ASR says they are calling out the BS in the industry but they have become the same demons they repelled. ASR promotes a bunch of mediocre sounding Topping products as world-class.

4) We have good sounding products and you measure it bad

5) You guys measure bad sounding products and say it’s amazing.

Hear me out, what if a poorly measured product is actually a great technical product and hence it sounds good? That’d actually make more sense.

@mdalton Providing consistent measurement for DACS, but the data doesn't seem useful. I or you can't look at the data and say this better measured one will sound better. Which is where I question its usefulness. If I can't predict the sound quality based on the measurement, then what good is the measurement?

@laoman Interesting product. Thanks for bringing it up. Bruno is a measurement-first engineer. Anything he makes is guarantee to measure well, but is not a guarantee they will sound good.

There is so much discrepancy between how the Tambaqui sounds, vs its pricetag, vs its measurement. I’m gonna make an argument and say it’d have made more sense for everyone, Amir included, if the Tambaqui measured bad instead.

Full disclosure: I have not heard the Tambaqui

1) Based on owners impression and reviews. The Tambaqui performs on the level of Chord Dave, and DCS Bartok. The pricetag reflects their performance as well. Chord Dave - $14,400. Tambaqui - $13,400. Bartok $20,950. Bartok is 50% more but I digress (I've been told Bartok used to be very close in price to the Tambaqui).

2) Bartok measures BAD. Dave measures BAD. Tambaqui measures GOOD. Huh?

3) Topping D90se - $900. Measures GOOD. It measures so good that it is nearly identical to the Tambaqui. $900 vs $13,400. Nearly the same measurements. Huh?

4) I’ve owned the D90se. It sounded bad, subjectively bad. There is just no way the D90se would sound as good as the Tambaqui despite the measurements. The measurements for these 2 products make no sense, no sense in price, no sense in performance.

So to conclude, the measurements make no sense, Amir once again proves his data is meaningless. 3 products of similar performance, 2 measured poorly, 1 measured great. Makes no sense. Logical conclusions cannot be found at ASR nor from Amir. The only thing that made sense here is the pricetag (kind of).

 

This is what Amir had to say at the end of his Tambaqui review,

"Since I am not the one paying for it for you to purchase it, it is not my issue to worry about the cost. As such, I am happy to recommend the Mola Mola Tambaqui DAC based on its measured performance and functionality."

He’s happy to recommend the Tambaqui when the D90se can be had for $900. Such a humorous clown. 

Laugh out loud, roll on floor laughing again until I poop my pants. 

@mdalton I only responded to 1 of your 3 examples because I am very unfamiliar with the other 2.

All of the 3 mentioned dacs are exceptionally clean and have exceptional detail retrievals. Which makes no sense because Amir measured the Dave to be very poorly.

@mdalton I sense a bit of disappointment in you in that I only responded to the dac section. I’ll talk about the power conditioner part here. I’ll do you a good one.

The Puritan 156. A HIGHLY beloved product among audiophiles. How does Amir measure it? Well let’s quote Amir himself.

Conclusions
As you see, I have run a number of tests to give the PSM156 ample opportunity to show it can do something to improve audio but it can’t even move the dial one hair. There is no indication or logic that would tell us that it can make an audible improvement. Yet the company says this in the feature list of this product:

The only thing reliable about ASR is severe inconsistencies, which is exactly what good measurements avoid.

Calling ASR a cult and selling snakeoil is out of good conscience, a very accurate description.

I don’t see the purpose of ASR existing anymore. If you believe in ASR’s measurements, You should only buy Topping products. Get their dacs, amps, and preamps.

There is no point in measuring X products. Just get a Topping and be done. Which Toppings don’t matter either. 123 sinad is not going to make a difference vs 124 sinad.

Buy Toppings, Buy Benchmark, Buy anything that Bruno puts out. These 3 are going to put out SOTA measurement products.

@laoman His (oberoniaomnia) post just proves ASR measurements make no sense. 99% of R2R Dacs measure terrible vs the Delta-Sigma Topping dacs.

For him to even say the Topping and Holo May are more or less equal is just wild. Because the measurements wouldn’t support his statement. Keep in mind how good his Topping D50 v.3 measures. It should on paper, poop all over the Holo May.

I’ve previously stated the Topping D90se (previous Topping flagship and a measurement beast) is subjectively bad.

1) Small soundstage

2) Very poor actual dynamic range. Let’s say you have 10 different sound in the Loud range, 1-10. The Topping can only do 5-6-7 for all 10. The same for problem for soft notes.

3) Poor spaciousness. Lack of air and space between instruments. The small soundstage makes it worse.

4) A very homogenized and boring sound. Because of #1 #2 and #3, Most instruments and notes have similar loudness, It just sounds very similar tracks after tracks. At the end of the listening session It would feel like 1 very long track.

5) Flat dimensionality. Clarity is high but transparency is low. Everything is upfront,

Compared it to my old Anedio D2 dac ($1,200) at the time. The Anedio is leaps better in every aspect, except vocal’s warmth. If I didn’t pay for the Topping myself I would’ve thought it was a $300 dac. All of these are subjectively bad traits in audio. Congrats to D90se for being a measurement beast and sounding the way it does.