Amir and Blind Testing

 

 

I just did a blind test between my Benchmark LA4 preamp and my Conrad Johnson premier 16LS2 tubed preamp.

Blind testing can be a hassle so I indulge in it fairly rarely, but I've done a few over the years.  Fortunately the Benchmark preamp made a blind test pretty easy in terms of matching levels and quick switching (with the help of my son - we randomized switching etc).

Results of the first test were an easy 15/15 correct guesses as to which preamp was playing.   I'm still a fan of toobs!

I said we must listen any produtct in good acoustic condition...not only read some chosen electrical measures to assess his quality...The final quality of any audio system is DETERMINED by his relation to a ROOM acoustically controlled...

is it clearer?

 

Not if you are making technically implausible claims...no.

In many instances (not all, but many) we really can say, through measurements, that "X is bad" or "the difference you heard was in all likelihood your imagination."

It doesn't matter what room you listened in.   That would be like saying "I saw a working Perpetual Motion Machine!"  A physicist explains why that is impossible and you reply "But did you test it in the guy's GARAGE where I saw it?  If not...your opinion doesn't count!"

 

 

 

What a load of nonsense. Amir has no agenda. There are a wide variety of people who participate on the ASR forum from subjective tube amp users to nothing but the highest measurement users. I take that back there is an agenda exposing bull products like "audiophile switches, cables, fuses " and other highly questionable devices that serve no purpose . Get over yourselves and yeah, hopefully younger people getting into this hobby will have a bit of common sense and see BS $20K DACs and $10K cables for what they are jewelry.

 

So true.  The amount of nonsense, strawmen and misunderstanding spewed about ASR could fill libraries at this point.

Some posts here amount to "why can't we just go on letting young audiophiles be duped by manufacturer and dubious audiophile claims like the rest of us?"

Some of the claims in high end audio are a balloon, ASR is the pin, and some people see their own beliefs being burst and don't like that feeling.

agisthos,

 

Amir is the high priest of the ASR cult.

^^^ Not a very promising start.

Years ago, they were militantly claiming that changes to the power supply of dvd/blu-ray players could not possibly have any effect on the video output, because bits are bits and the power supply does not effect the bits.

Given HDMI has been the default transmission method for many years, that claim is correct. Luminance information, like color and other picture values, is encoded in the digital signal.

It is NOT determined by the power supply to a blu ray player. A pixel on the display either receives the information or it doesn’t. (And when it doesn’t, artifacts can occur but they are not of the type you are describing - a rise in brightness level or color saturation of the displayed image).

 

It was a theoretical argument,

LOL, no it wasn’t. It’s literally how digital video information works - which is shown in practice millions of times a day (at least). If it didn’t, we would be having a heck of a lot of image problems that we just aren’t having.

 

which was easily disproven by doing any changes from SMPS to LPS, which on the meter would measure visible changes in brightness levels, let alone color measurement differences and detail improvements.

I call B.S.

Show me the evidence you are speaking of where (I presume using HDMI) changing the power supply to a blu ray player measurably altered the brightness, color etc of the image.

I’ve been a member over at ASR forum for 20 years, where professionals and dedicated enthusiasts have been exchanging notes on calibration - using all sorts of sensitive equipment - and NO ONE has reported anything like that, nor any measurable changes between two properly functioning HDMI cables. Because....that’s not how it works.

 

And it’s just this type of nonsense that we can be thankful there are people like Amirm around to test.

 

 

 

 

Amirm is doing a terrific service to the audiophile community.

You don't have to bow to him as a "cult leader" (ridiculous meme going around) or believe everything he says is beyond critique.   His reviews are quite well vetted even on the ASR forum, where his methods and conclusions are examined.

But taking on manufacturers claims and to the extent possible examining what objective evidence can be gleaned or not in terms of measurements is providing one hell of a lot more information than we are finding in most other places (manufacturer's marketing most of all).

There is a part of the Golden Ears crowd who have an almost mystical belief in their own incorruptible, observational powers - "science can't tell me I'm wrong!  I EXPERIENCED IT."    Those people will likely never be reached.  

 

And of course as I've argued many times, no audiophile has to pay any attention to measurements or science - we can all buy whatever we want, for whatever reason.

But Amirm is providing some extra information for those who seek it, which is great.

 

mahgister,

Thanks for alerting me to a typo.

I meant to write:  "I've been a member of the AVSforum forum for 20 years..."

(Not ASR).

 

 

No problem, invalid.  It's just that I couldn't go back and fix the typo in the original.

invalid, my subsequent post mentioned it was a typo.  See the correction.

What people like @Prof are ignoring is that according to PS Audio, the fake science site (my term) measured the wrong AC output. Has anyone seen them publish a revision or retraction?

 

I’m not ignoring anything: it seems you who is leaping to uninformed conclusions.

Did you not see that Amirm did a follow up review taking on all the issues that cropped up after the first one?

The problem wasn’t Amirm; it was that PS Audio’s own user manual made no mention of a difference between the ports, that one was limited in any way, and Amirm did what any user would be likely to do: use the port labelled ’High Current" for their amps. Nobody could fault Amirm for not knowing information PS Audio themselves didn’t think to put in the user manual! In fact it was due to Amirm’s review in the first place that PS Audio customers are now alerted to the fact plugging their amp in the High Current outlet can reduce amplifier power!  That's not info PS Audio was telling their customers for that model!

Then Paul came out saying Amirm should apologize for the mistake...while Paul himself was stating misinformation about he product (as shown by Amirm’s measurements) and Paul later admitted screwing up.

In any case Amirm DID take all the brew-haha to heart and did a follow up review addressing the criticisms. Spoiler alert - still no sign the PS Audio device would do what it claims for the *sound* of most amps:

 

A blind listening test is not a test for bias.
It shows that listening under those conditions is mostly more difficult, but it does not explain WHY it is more difficult.

I’m unclear on what you are saying. On your account: What about "listening" is "more difficult" under blind test conditions?

 

Also, why would Amir not be biased himself? Maybe he does not want to hear (or accept) a difference?

Especially when it comes to vetting a controversial technical claim for audible differences, Amirm’s sighted impressions are no more reliable in of themselves than anyone else’s.

That’s why in principle anyone who BELIEVES he hears a difference could test themselves (using blind testing to help rule out sighted bias).

That said, Amirm’s sighted impressions are at least backed up pretty heavily by the objective evidence he presents. Unlike the usual audiophile anecdote or audiophile marketing.

 

Also, as we still not completely sure WHY we experience differences in cables, who is to say he is even measuring the right parameters? What if our brain can detect the differences? How will we measure that?

To be clear: I am not saying he is wrong, but I am suggesting that there are too many questions left unanswered to conclude that he is right.

By that measure you couldn’t establish any conclusion!

The best we ever have is what the evidence suggests.

First of all, we can test whether your "brain" can detect any differences via blind testing. If sighted bias and knowledge are ruled out, and you are ONLY able to use your hearing and you can not reliably detect a difference between A and B, then it’s reasonable to assume you aren’t detecting any difference.

This is how standard audiograms/audio tests work for hearing, right? If you can’t reliably detect tones over 12k, guess what? You can’t hear over 12k even if you CLAIM to or believe you can or not.

Similarly all sorts of tests have been done to establish the parameters of most of our senses, including hearing. Once you are down to a certain level of distortion, nobody has shown they can reliably detect it hence no reason to think we can hear it. It follows that if you measure a device and the distortion levels in various relevant parameters don’t rise above the known audible threshholds, then the reasonable conclusion is "that distortion is not audible."

We aren’t beholden to any person’s claim to have Super Human Hearing..unless they can demonstrate it in similar controlled tests.

The engineering involved, the principles on which USB cables are designed, tells us that any competently designed USB cable will transmit the 1s and 0s just fine, especially when used with a competently designed DAC.  And that heroic measures beyond that are rarely needed (and in any case, would be unlikely to be responsible for the sonic claims made on behalf of those cables).

Amirm tested for ANY way the Nordost cable altered the signal, looking for any relevant distortions, and there was nothing relevant to distinguish it’s performance from a cheap amazon cable. Before someone says "But we can HEAR the difference so maybe he’s measuring the wrong thing" you should be able to DEMONSTRATE you can hear the difference to take that claim seriously. And that would be under conditions controlling for your knowledge of which USB cable is being used. Otherwise it’s just begging the question.

 

 

 

 

 

 

emailists,

 

It appears you have not gone through that second review.

Amirm DID originally do tests on the other Zone A outputs, but only used the last outputs for the amp on the very rational inference that since they were labelled "high current" those would be best for powerful amplifiers.   The owners manual tells the user the voltage output is the SAME ACROSS ALL ZONES so there is nothing to suggest any problem with using the zone D outputs.

It's entirely PS Audio's fault for this lack of information for the user.

Paul McGowan didn't help with information by claiming the D zone didn't regenerate power, so Amirm shouldn't have used it.  Amirm's own measurements showed Paul was wrong, that it DID regenerate power.  Paul later admitted to the mistake.

So the source of accurate information here has been Amirm's measurements, not the user manual nor the company spokesperson!

No user, or prospective customer, would likely know any of this without Amirm having measured the unit. 

Further, as I understand it, PS Audio never reached out to Amirm to try to "correct" anything in the review.  Rather, Amirm was alerted to some dialogue in the PS Audio forum criticizing his review.  So he took it upon himself to take another look addressing the criticisms and the additional claims made by McGowan in the thread.   THAT is actually being a conscientious reviewer!   But of course you apparently don't want to give him any credit possible.

Plus Amirm showed that even the other zones regenerating power did nothing to help with amplifier power/dynamics or anything that would support the company's claims for how it will elevate the sound quality of your system.

It was completely predictable that PS Audio's damage control video would miss the point.

These companies always make claims that attaching your gear to their product "dramatically improves the SOUND of X, Y and Z in your system."

But they never DEMONSTRATE any measurable - or audible - change in the SIGNAL COMING OUT OF GEAR ATTACHED TO THEIR PRODUCT.

It doesn't matter much if you are "correcting" for problems that are either generally corrected for in the design of lots of gear (e.g. amps) or that are not audible in the first place.

THAT was the point of Amirm's review.  Amirm showed that, yes, the power plant DID regenerate and regulate the power IF YOU JUST MEASURE at the power plant.  The question was "But what BENEFIT are we likely to see when you attach an amp?"    That's what Amirm did, showing there was essentially no benefit likely to make any audible difference.

This is, of course, precisely the issue the new PS Audio video avoided.   They just did the usual "Look how our device alters the power" and then IMPLIED that this has the audible benefits they claim.  It's that last bit they are missing.

Prof, with all due respect, Amir’s approach still assumes that he is measuring everything that matters in terms of ultimate sound quality. I think that is unlikely to be a valid assumption. If Amir’s assumption of perfect measurements is invalid, then the importance of human listening cannot be excluded from any evaluation of a product designed to improve the sound quality of reproduced music.

 

Amirm is coming to the question with knowledge about what type of distortion is possible, and what type of distortion is audible (given known thresholds of hearing).

If there is some OTHER form of distortion Amirm is not measuring for that would be audible...what would that be?

Shouldn’t PS Audio have identified just what that distortion was...if they even came up with a solution for it?

Yet PS Audio isn’t telling us they’ve identified some unknown type of distortion. They are saying their product fixes the AC power in standard ways that any engineer can understand...and test for! Which Amirm did. Amirm tested the effects of the regeneration on the power AND the effects with an amp hooked up.

What else could he possibly test? Simply waving at Some Mysterious Yet To Be Discovered thing no more helps PS Audio claims than it does anything at all.

As to Amirm not performing blind tests, I agree those would be a nice addition.

But given the objective evidence already showed distortion differences known to be inaudible, there isn’t any good reason to presume they’d be audible in blind tests.

PS Audio could, of course, always show otherwise. But we know they won’t. (The best inference being: because they couldn’t demonstrate any mysterious unmeasured audible distortion).

 

milpai,

 

I think he is measuring the wrong thing. Ask your guru to show how he measures a person's emotions. Some folks prefer precision while some prefer musicality.

 

That's missing the point.  In the case of, say, the Nordost USB cable or PS Audio P12 review, Amirm's measurments indicated no change to the signal that would be audible at all.   Measuring someone's "emotional response" tells you nothing about what's actually happening in reality, in terms of the gear.  I don't give a damn if you have an "emotional response" to a Nordost USB cable because that's you (and likely your imagination).  I want to know if it ACTUALLY does something for the signal so I know what I'm spending my money on.

 

Some approach high end audio like they do a religion.  Not everyone wants to do that; many of us want actual knowledge as to how the equipment works so we are making informed decisions with our money.

 

 

I'm ready glad that audio precision doesn't think like the objectivists on this forum, they actually think there is more or better measurements that can be performed in the future to better understand what we are hearing.

So you are against the idea of searching for a better and more reliable understanding of what we are hearing?

That sounds like flat-out anti-science.

I'm glad there are people out there trying to investigate these things, rather than

being happy with our current state of understanding.

I think this is now the 3rd time I’ve posted a link to this topic over the past 2 years concerning ASR so here it is again.

 

ASR is pretty used to empty responses like that one. It basically says "I don’t actually have any good, civil arguments or evidence in response to ASR’s reviews...but since I still don’t like their conclusions...here’s a disparaging meme so I can feel like I got one over on them."

 

Embarrassing enough once. But..3 times?

 

Nonoise,

I apologize.

I did not notice that there was a link associated with that picture. I’m so used to seeing ASR utterly dismissed, especially by those who take the more subjective approach, that just seeing a picture "Bullsh*t" seemed more of the same.

But clicking the link I do see the author of that piece wasn’t just brushing away ASR but attempted a more detailed case against some of ASRs claims. So I was wrong in assuming you were just posting a picture. Sorry about that.

 

(A quick skim of that link suggests I don’t agree with some of it, I’ve had my own clashes with ASR and there is a sense in which some there "take things too far" IMO).