Well, I am very curious to hear any system that was assembled through blind testing of its individual components....
Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)
About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.
Yes, each meteor produced a sizzling sound, like the sound of a frying pan.
Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.
ASR would have agreed with this sound reasoning based in elementary science.
Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.
Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.
I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.
406 responses
This is bias likely derived from reading, not knowledge or experience. Your purported expertise is recording? Mixing? Audiophiles often write about how superior audio recording and mixing used to be, even in the 70's into the 80's. Do you think most mixing consoles used op-amps, or discrete transistors? Can you clearly communicate what you think op-amps and negative feedback is doing to the signal, and why this does not show up even in complex distortion measurements?
|
Or...many people are triggered by having their own subjective conclusions challenged. "How dare Amir tell us what we can and can not hear!" It's taken personally, so often the response is to go ad hominem on Amir, call him names or dogmatic etc. When often it is the more "subjective-based" audiophile who is closing his ranks around his beliefs. Amir has actually responded quite civilly to those engaging him likewise. And his response was even quite measured to those implying he is dishonest or colluding. I certainly don't mean to paint anyone as a saint, Amir included. But there is a lot of assumptions made that I find come from a bias on the "subjective" side, where they see what they think are pushiness from ASR, while turning a blind eye to the type of ad hominem and insults from "their own side."
|
The same could be said about non-audiophiles, who don’t even bother putting the amount of effort you have in to what they listen on. People just listening in laptops, iphones, earbuds, enjoying music and getting on with life. But we audiophiles care about sound quality, and are also fascinated by audio gear (which is why sites like this exist). The degree to which anyone is fussing over something will vary among individuals and have little to do whether one is in the "measurements camp" or the "listening only" camp; it will depend on the individual or even what particular stage they are at (e.g. in "upgrade/auditioning equipment" mode or sitting back enjoying the system mode). So, sure you can find ASR members who spend a lot of time fussing over measurements (doesn’t mean they aren’t enjoying their system too). You can also find plenty who just bought some gear based on the available measurements and...that’s that. They just sit back and enjoy. Alternatively you can find plenty of "subjectivist" audiophiles who endlessly fuss over their speakers/amps/DACs/tubes/cables etc "breaking in" or all manner of tweaky stuff that you won’t find ASR members worrying about. And you can also find more subjective members who aren’t fussing like that and are just enjoying their system - some change their gear a lot, other’s don’t. So there isn’t any in principle sense in which looking at measurements means someone isn’t able to just enjoy their system. You may enjoy reading a subjective review about equipment, someone else may enjoy reading how a particular thing measured. It’s all good. |
I think it is a hard concept (or fact) to accept, that for some, deep in whatever aspect of the technology we may be in (for me speakers), that we can get a lot more useful and unbiased information about how something sounds from a very detailed set of measurements than we can from someone (not us) doing a listening review of a product. I can't speak for others, but I expect in whatever product they are experts in, that they also can get as much or more useful and unbiased information from a detailed set of measurements than they can from a listen only review done by someone else. |
@andy2 ,
What did you want us to take from that PS video article? One would not measure interconnects by measuring the output of a speaker. Speakers are too sensitive to environmental conditions, prior operation, etc., not to mention that would also introduce an indeterminate error that may also have time effects from the speaker/amplifier interface. Far more accuracy would be achieved by measuring the output of the amplifier without the speaker which is what Bob Carver did when the made the SS amp match the output of the tube amp. A little bit of Paul's bias shining through when he said made the SS amp sound as good as the tube amplifier. |
No, you worry about a ton of things in your system that don't matter while we enjoy music. You think your wires may have sound. You think your amp has sound. You think the table you put the system on has sound. You think your AC has sound. You think digital sources have sound. You think, well, you get the point. We on the other hand, buy performant systems with confidence and sit back and enjoy it. We know why it sounds right. You don't. You are forever chasing ghosts in audio. The anxiety that comes with that must be immense. Ask anyone who has converted from your camp and above is the answer they give you. While you keep upgrading, tweaking, replacing stuff to remove that other "veil" and get blacker backgrounds, we queue up another track to enjoy. So I suggest getting off that talking point. That dog don't hunt.... |
You could do that. Or, if you are in our camp, use measurements to rule out audibility in many components (i.e. they are transparent). For others such as speakers, you can rely on companies that perform double blind tests, or use research that correlates what sounds good to us with respect to measurements. I have done this across some 200+ speakers now. The research works wonderfully. Same mostly works for headphones as well although measurements there are subject to more variations than speakers. Remember, the job here is not to give you 100% answer. It is to get rid of 90% of the variability by weeding out clearly broken and non-performant gear. The rest you can choose from and take in factors beyond performance. Compare that to the alternative the few of you follow. Completely unreliable listening tests. 1000 and one opinion about every gear, every cable, everything you can name. True wild west with zero regard for decades of research into what makes sense. |
You are a fountain of untrue assumptions. Do you even bother to fact check anything before just throwing it out at us? Didn't I already show you how proper, peer reviewed research shows audio reviewers to not be remotely capable of producing reliable assessment of speakers? Here is Dr. Olive's research again:
Do you see how Audio Reviewers are even worse than audio sales people? If they are reliable as you say, how come they failed so catastrophically here? They failed because they don't have trained ears despite all the gear they have listened to. That you declare them to be reliable means that you have not spent any time reading and understanding this topic. You continue to shoot from the hip, throwing claim after claim without an ounce of proof. All lay intuition meant to defend your position in audio. Spend less time here and more time reading and learning about the topic. Here is Dr. Olive's blog if you don't know where to get the research papers:
Or watch my video where I go through this and explain it in understandable manner.
|
Whether or not we agree entirely with @amir_asr ’s perspectives (I do not, although I understand the rationale and logic), he is taking the time to present those perspectives and back them up, on a forum that is not very friendly to his perspective. I didn’t expect Amir to engage in this discussion, in a community that rejects his perspectives with a degree of hostility, so I commend him for presenting his views in this environment! I for one am reading his links, even if I still don’t agree fully. But becoming educated on other’s perspectives never hurts. In some cases I can even be convinced to change my perspective. But I don’t think that will happen in this case - after all, remember, for 25 odd years scientific researchers assumed that reports of audible meteors were figments of the observer’s imagination! |
So...because you can point to something science might have gotten wrong, the lesson is...anything goes? If someone makes a claim that is, on the basis of the current science, outrageous, does the fact science has been wrong at points mean there’s no basis to doubt anti-scientific claims? See...this is the mushiness of such a position. Perpetual Motion Machines are impossible based on current understanding of physics. Yet someone claiming they know a guy building Perpetual Motion Machines in his basement can also say "Don't be so skeptical! Remember...science has been wrong before! Remember the meteors!" Does that for even a moment give more credence to the claim someone is really making Perpetual Motion machines? Of course not. You can’t use anomalies where science got something wrong...which by the way is always corrected by science!...as if science doesn’t really have an excellent grasp on many things. People are ripped off every day of the year by people selling things with the tag line "It Works! And Maybe Some Day Science Will Catch Up To Our Discovery. They Called Galileo Crazy, Remember!" The reasonable approach isn’t to believe something dubious "because maybe science got this wrong." It’s rather to wait for solid evidence, if necessary scientific evidence, showing THAT phenomenon is real vs the countless propositions that are false.
|
Science is always self-correcting. Just look at the things that have been accepted as true, only to be rejected later. It was only recently that we are questioning the general advice that a moderate amount of red wine is actually good for you, to name one example. The meteor analogy is one example of many. But the analogy also points out that we may not be measuring the right things to explain our perception of physical sound waves. I think this is actually the larger point, as science can only measure what we know. Based on my own subjective experiences in this hobby, which admittedly is flawed due to being subjective, I have experienced enough to personally believe that there is far more to learn about how we perceive sound waves, and about how we measure to accurately predict what we will perceive. If I suddenly saw a ghostly apparition appear (for the record I’ve not seen one), and I wasn’t on any mind altering substances or expecting any sort of psychosis at the time, I’d probably form the belief that ghosts were possibly real, even if science had any proven it. In HiFi, I HAVE experienced the unexplained, albeit subjectively, but this is enough for me to form an opinion that maybe measurements are not sufficient to explain what we are experiencing. |
Just to give one recent example to relate to my prior post: Last year I bought a second USB cable (Audioquest Diamond) while I had the original for a few years. The new one sounded inferior. To the degree that I wondered if there was a design change or changes to production. But after a few hundred hours, I could no longer tell them apart. Yet existence of burn-in is endlessly debated. Another recent example is what I found with length of USB cables, where a Nordost Valhalla 2 2m cable sounded superior to the equivalent 1m cable. Intuitively I would have expected the 1m cable to sound better, as I had not at that time read the theories, to my knowledge unproven, that USB cables should be longer than 1.5m to accommodate “reflections”. And I don’t know if this one is proven or if people just have theories grounded in science, but the whole anti-vibration/isolation tweaks that really do make a surprising difference. I introduced a friend who is newly into HiFi to Herbie’s Tenderfeet, and he promptly cut up some yoga mats to replicate the benefits himself. He tells me it worked! It’s easy to identify whether the feet were added or removed, at the transition points. I wonder, would @amir_asr be able to measure the differences I’m hearing, in each of these example cases? Not a rhetorical question; I’m genuinely curious. If the answer is no however, I think the assumption might be that it’s all in my mind. Just like what the researchers said about those hearing meteors, 25 years ago, because they were not measuring the right things or applying the right science to explain our perceptions. |
@amir_asr Regarding system assembled on blind test methodology... You do not have to go so far, you can simply set up system, based on components that you recommend and publish it. It should not be difficult to replicate it and to hear first hand what (for you) represents the ’good’ sound. I agree that lots of hi fi gear is overpriced and many simply does not sound good, but its the same thing with various different products, cars or ’wonder’ diet pills, or whatever else. But, that is common knowledge and everyone is trying to find the best value for its money, or his needs. Even among people from the same camp, aka the ’subjectivists’ it is often very hard to find consensus for many things. Building a great sounding system is a sort of an art form,put ’wrong’ cable on a ’wrong’ place and its ’arrivederci’ Roma (sound) In the same time and please dont take this personally, I am surprised that there are people pretentious enough, who are trying to convince others that their choices are the ’right ones’. But, than, why stop only on hi fi? I am sure that there are more interesting challanges, or more noble ones? As for your ’camp’, the prevalent atmosphere on Asr forum scares me. Or amese me, but not in a nice way. Owning tubes, vinyl, cables, or anything ’expensive’ is potential health hazard if one finds himself surrounded by that bunch. What I really do not undersatnd is why so many people refuse to trust their ears and why so many people need dogmatic ’guidance’ ? But, silly me. Everything that is happening in our world, on much larger and ominous scale shows us how the mass psychology works. Pity that even a simple hobby, idiosyncratic as it might be, can not be spared of such folies
|
Did the cable change or did you change?
>1.5 meters is for SPDIF, not USB. |
@thespeakerdude , the cable changed not me. How do I know? Because I had a control (the original cable). And actually the >1.5m is the guidance for USB according to some - pretty easy to google. Even Mark Coles of Sablon confirmed to me that he’s heard that as well (he said he hadn’t heard that for AES/ SPDIF). But as I mentioned I don’t believe this is “scientifically proven” anywhere. As a side note I repeated this finding with a .7m Audioquest Diamond USB cable that sounded very compressed and closed in compared to the equivalent 1.5m Diamond. I even preferred a generic USB cable to the .7m version of the Diamond. Not that vendors are to be believed, but Nordost has minimum cable lengths listed on their FAQs and suggest >1.5m for all digital cables. I wouldn’t have put much stock in this if I hadn’t FIRST encountered this in my own trials, without having any expectation at all being ignorant of the guidance at the time. These and the other HiFi phenomena in I referenced are differences I know beyond any doubt (to myself, not for others of course!) that I hear. My mind intuitively expected the shorter USB cable to sound better, and I found the opposite to be true. I know the argument remains unresolved - are these things in my mind, or is it that science has not yet figured out how to measure certain things that we perceive in audio? It’s not really an argument that can be conclusively won in either direction in discussion. After all, how can one prove the existence of something we don’t yet know? But I think it’s a good discussion.
|
It's amazing how our senses can sometimes pick up on things that science can't quite explain yet. It's great that you trusted your own perception and kept an open mind, even when scientists were saying it was impossible. And now, with new research, it's clear that what you heard was real! This just goes to show that there is always more to discover and learn about the world around us. Keep being curious and open-minded, and who knows what other incredible experiences you might have in the future! |
I’m not sure if you meant it this way, but your post could imply that a "skeptic" or "objectivist" is the one not having an open mind. To be clear, that is far from the case. Whether it’s Amir or me or some other like-minded folk, we are open to ANYTHING that is true! That is the whole point of doing rigorous inquiry. We simply ask for good evidence for a claim, especially if it’s an extraordinary claim relative to current generally accepted theory or practice. Given that countless wild claims are made every day, what other approach could be more reasonable? One should always be "open minded" in the sense of being open to any evidence for some new thing, and ready to overturn any current beliefe we have based on good evidence. But that should always be in the context of how plausible a claim is, and what type of evidence has been provided. We have built bodies of hard-won...and PREDICTIVE knowledge....by being very, very careful this way. If "open mindedness" is not tempered by critical thinking this way, then being "open-minded "may as well be a synonym for a lack of critical thinking about claims - to be vulnerable or gullible. For instance, if I claim I could sell you a perpetual motion machine, which would solve your energy bill permanently, how "open minded" do you think you should be to the truth of my claim? Wouldn’t your skepticism...and a high demand for evidence, be quite warranted? It is a mistake to presume skepticism or demands for good evidence equates to close-mindedness.
|
https://www.theaudiobeat.com/visits/shunyata_visit_interview.htm (This link comes from a discussion here at Audiogon). I think Amir can learn a thing or two from the link above. It actually measures transient affect of power cable. I think Amir mostly measure frequency response which is steady state. A lot of thing happens in transient. |
Your surprise should be directed at whoever created this thread, directly challenging what we do at ASR, and manufacturing comments about me and what we do to boot. And those of you flagging me personally, causing the form software to send me a notification. I should ignore it but then folks go on and on here, piling on falsehood on top of falsehood. A fellow here even went after my personal career! As to your car example, no, it doesn't apply. There is a ton of scrutiny of car performance through measurements such as 0 to 60, breaking, cornering, etc. from press and online reviewers. As such, a car company can't claim that it has built the fastest car in the world where in reality, it is slower than a Honda Accord. Yet, that is what happens in audio. Until we came about, objective evaluation of audio was delegated to an appendix at the end of a few reviewers and that was it. Yet, every audio company claimed to recreate reality of live music, etc. We get started and shine a light on equipment performance and folks are up in arm. How dare you do this and give audiophiles more information? As you say, demand is made to "leave them alone!' Personally, I do leave them alone. We have a great audiophile society locally and you don't see me going there trying to change the opinion of many subjectivists there. Thankfully many come to me at meetings and ask me questions. So please don't pull that stunt and debating tactic at me. The reality is that you are bothered by what we do so you want it stopped. Well, it can't be stopped. People are seeing the value of reliable information about audio and are abandoning what some of you have taught them. Not everyone of course. But we are not running a sprint but a marathon. And not trying to boil the ocean.... |
If that were true, Amir would be sure to listen to every component that he "tests." That he sometimes manages to avoid listening - and has a whole bunch of wordy rationalizations to justify that - undermines whatever science he’s trying to pursue.
This is a hobbyist’s group, not a scientific forum, so your "demands for good evidence" really don’t belong here. If you don't like the evidence presented, it's really your problem. No one here owes you anything. |
Did you know which cable was in the system, or did you have someone switch it such that you did not know?
The 1.5 meter is definitely for SPDIF and there is a good explanation for it. I have a formula written down. (Rise time)/(1.5 * 3) in feet. SPDIF rise time first google hit = 25nsec. Cable = (25/4.5) = 5.5 feet = 1.7 meters. USB rise time 0.3nsec first google for 2.0. That is 0.3/(1.5*3) = 0.07 feet. I think if a cable supplier has not heard of the 1.5 meter thing with SPDIF, you need a new cable supplier. Maybe you were having some noise issues? |
cleeds, Amir does listen to much of what he reviews. You know it. And in the instances he does not, when you say Amir merely has a "wordy rationalizations to justify that - undermines whatever science he’s trying to pursue." That is a completely unsupported claim on your part. He may well have a perfectly good justification. It’s like if you claimed that I need to come to your place to see your Perpetual Motion machine, and if I point out that current understanding of physics suggests your claim is so unlikely it’s not worth my while to make such a trip. The reason for not bothering to "go see your perpetual machine" would BE based on currently understood science! You could try to dismiss my reasons as some "wordy rationalization" but that does NOT actually address the REASONS. It’s just a dismissal of an argument using a lazy characterization rather than an actual rebuttal. Likewise, if Amir doesn't listen to a certain component after measuring, he may well be justified based on all sorts of studies and science that tell us "X won't be audible." If you want to point to specific examples to make your case, and you can show why Amir was unjustified in not listening...be my guest.
|
That's the logical fallacy of the Exluded Middle. Don't be silly.
I've already done that. In my view, a test of an audio component is not complete without a listening evaluation. I understand you disagree, and that Amir disagrees vehemently. So be it. |
https://www.theaudiobeat.com/visits/shunyata_visit_interview.htm
All I get from this @andy2 is that you can fool Michael Fremer all of the time. I will defer to @amir_asr who I believe is an EE, but even I can tell that article is just blowing smoke. An honest article would hook up the power cable to an off the shelf unmodified audio product and show the measured waves with AC input not a fake input. |
@prof Amir listens all right, like when he listened to one Magnepan LRS, after he measured them like they were a small bookshelf speaker. |
No it's not. It was an example of a lazy argument I was flagging one not to fall in to, and therefore leaving it open to you to justify your position. I didn't foreclose that you had a good argument; I asked for the argument.
Sure, that's fine. Except that opinion offers no justification for your characterization of Amir's position, re any specific instance where he didn't listen. It still hangs in the air looking like a lazy dismissal, rather than engaging his reasons. Hence...my previous example remains relevant.
|
@amir_asr wrote and I quote 'So please don't pull that stunt and debating tactic at me. The reality is that you are bothered by what we do so you want it stopped' Dear Sir, I was just beeing polite and answer to your comment on my previous post. I do not see any subject that we might debate over (cars included) and certainly nothing bothers me about your hobby. Its mystery to me if and why anybody is? But, as you have mentioned cars and audio, you have missed the crucial point. Performance of cars can be messured, bot not the feeling of driving one. Same with audio, despite all marketing or messured perfromances, the feeling or recreation of some percived musical event is something that is happening inside our heads. For that reason there are so many different visions of sound and that is why we have Franco Serblin and Magico, set tubes and monster ss, etc Now, I do not believe that you have not realised that before, its just something inside you that makes you want spread your black and white vision. Nothing wrong with that either, but just dont stop with audio. You could do so much more. I urge you to reconsider possibility to expand your field of work. There is a whole world of windmills out there. You are perhaps suffering from illusion of your own 'grandezza', but without it you would not even started doing what you do. So, go ahead, noble ingenioso hidalgo, the world needs salvation |
| Post removed |
You could convert the phase noise from say a 1kHz tone to jitter in time domain, but jitter measurement is easier to measure in time domain. Measuring phase noise will need a fairly expensive piece of equipment to do it right, but a more affordable real time scope can do the job better. Also to measure phase noise you need a pure sine wave. But in time domain, you could do it using a square wave or some transient waveform, which is more akin to real music. |
| Post removed |
That will be impossible when you are measuring the output of the DAC and have no access to signals that are hidden inside a box or hidden inside a chip. The only way is to measure the analog output. Fortunately that is something that can and is done with very good accuracy.
|
| Post removed |
The specific instance - something we debated at the time - is immaterial. My opinion, which I need not justify to your satisfaction, is that a review of an audio component that doesn’t include listening is of little to no value. And we all know that Amir doesn’t always bother to listen, and has extensive, wordy rationalizations to justify that. He has lots and lots of words. |
| Post removed |
'My opinion (.........) is that a review of an audio component that doesn’t include listening is of little to no value.' In general: If everyone hears differently, shouldn't the listening to audio components not done by the person who is interested in a specific product (TT, DAC, amp and so on)? For example, would it be of big help for me if somebody tells me how a product sounds? I need to hear it with my own ears and brain in order to make a judgement. What good does it if somebody tells others how a component sounds? Or is the difference in hearing between people so small that it doesn't matter? So that we can rely on anybody's comments about the kind sound he is hearing? I do not know. Cheers, eagledriver
|
Here are some comments from a well known audiophile reviewer about a speaker he reviewed. What valuable information do you think you are getting out of this?
The reviewer is not talking about powered speakers. He is talking about passive speakers. Noise floor, passive speakers? Not the best thing to say to be credible.
He means front and 2-3 feet is not nearly enough to get rid of SBIR effects.
This is his listening room.
|
That’s for you to decide. Sometimes, as can happen with a movie or music critic, you find someone whose taste coincides with yours. That gives their opinion more weight. In audio for example, listeners may often independently conclude that a certain speaker tends to sound "bright," and that can be useful.
I agree completely, although I spell judgment differently. Of course, as has been shown in this thread, some dismiss those experiences completely. While that’s fine, it doesn’t obligate anyone here to submit to their demands for proof. |
