Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)

@andy2 

Amir has claimed that he personally listened to "200" pieces of equipment per years.  That is around 366/200 or 1.5 days per equipment.  I mean come on who in the world can take you seriously if you only spend such a short time evaluating an equipment.

In comparison, people at Stereophile spend weeks on an particular equipment before they publish the review.

They can spend months and it wouldn't make their reviews reliable.  If you know what you are doing, including science and engineering of the gear and what the measurements show, you can zoom in and find issues.  You don't sit there listen to random track after random music for weeks.  That tells you nothing.

Every one of my listening tests uses the same, revealing and proper tracks. I focus on what measurements say is wrong with the unit and test level of audibility.  A headphone amp that has too little power with high impedance gets tested that way.  And with content designed to find audible issues.

Same gear is tested by one of your favorite reviewers reads like a music review.  Oh listen to this album and that album.  What?  I want to know what the equipment is doing, not what music you listen to.  

Proper research has been performed to find such tracks.  See this:

You need to stop listening to your lay intuition and embrace science of how to do such evaluations correctly.  Formal testing shows long term listening to be much less revealing than instantaneous ones.  See this published research on that:

I implore to you start paying attention to decades of research on what it takes to properly evaluate audio gear.  The lay understanding and intuition stuff needs to go out the window.

@andy2 

That is because a good tube amp will cost a lot more money compared to a SS amp.  To get the same performance you need to spend quite a bit more.  If money is not an issue, most people would go with tube.

Back to the myth that money buys performance.  It can, but you only know it if you measure.

That aside, almost all content you listen to was created and approved by the talent using solid state electronics.  Is your claim that they heard it with haze?  If so, that haze must be part of the experience they want you to have!  Best to leave it just like guitar distortion.  :)

@amir_asr

They can spend months and it wouldn’t make their reviews reliable. If you know what you are doing, including science and engineering of the gear and what the measurements show, you can zoom in and find issues. You don’t sit there listen to random track after random music for weeks. That tells you nothing.

More time spent on a task can net better results. This is not always true. However, when it comes to listening (headphones, speakers, DACs, amps etc.) spending more time evaluating a product before releasing a review can help in a few key ways:

1) Testing for Reliability

2) Features & Functionality

3) Overall sound quality analysis

4) Small important details

I’m sure that most of these people doing reviews have a standard set of reference tracks; or at least a background/strong interest in audio; enough so to make their impressions reliable. They were hired to do a task and might be very good at it. We have no way of knowing how much audio knowledge they have...

Tyll Hertsens - of innerfidelity (now defunct) was probably the best reviewer of headphones on the net. Like I said in one of my discussions, he took time to describe what each headphone sounded like with a particular track. He also did measurements and highlighted key areas in the frequency response or octaves where performance could have been better...while still quoting the measurements he took. And on top of all this, comparing it headphones in the same price-range with the same factor - open/closed back!

Since he went above and beyond, this gives a potential customer huge insights! You can go ahead and test with the same track. You can wait a week, be really busy, visit an audio shop, listen to that headphone (or headphones) and jot down your impressions on the notepad app your phone has installed.

Then you can compare and contrast your review with his - figure out if this headphone is worth the money for you. I’m sure that anyone who has bought audio products based on solid reviews like his will agree...

Now back to reviews...I bought a SABAJ A10h based on your review. I also bought a DROP THX789 based on your review. In both cases, not only did I find that output power was severely lacking; each of them also had their own sound signature. Based on your measurements and overall write-up of both units, a potential buyer would actually believe that each of them were a wire with gain!

Please see my profile for a photo that illustrates this. Looking inside one of these devices tells you it is cheap to build. Uses an OP amp and tons of subtractive distortion limiting - like negative feedback in a circuit. Tons of this, much like dynamic range compression in mastering will limit perceived dynamic range in a track. You’ve got to wonder how they put something together at that price-point and sold it. You can easily look up parts by just looking inside a unit and doing a parts inventory check...they are both not state-of-the-art ! lol

Alright..you can have your cake and eat it too! All I’m saying is....live and let live. Your tone and how you almost bully people into listening to you is rather rude. Hence why virtually every audio forum on the web has labelled you all kinds of silly names.

You need to stop listening to your lay intuition and embrace science of how to do such evaluations correctly. Formal testing shows long term listening to be much less revealing than instantaneous ones. See this published research on that:

I implore to you start paying attention to decades of research on what it takes to properly evaluate audio gear. The lay understanding and intuition stuff needs to go out the window.

Well, I am very curious to hear any system that was assembled through blind testing of its individual components....

@alexatpos 

One of them probably has an answer to this too...

lol