Okay. I'll bite. What capacitor technology measurements should be improved?
IM Distortion, Speakers and the Death of Science
One topic that often comes up is perception vs. measurements.
"If you can't measure it with common, existing measurements it isn't real."
This idea is and always will be flawed. Mind you, maybe what you perceive is not worth $1, but this is not how science works. I'm reminded of how many doctors and scientists fought against modernizing polio interventions, and how only recently did the treatment for stomach ulcers change radically due to the curiosity of a pair of forensic scientists.
Perception precedes measurement. In between perception and measurement is (always) transference to visual data. Lets take an example.
You are working on phone technology shortly after Bell invents the telephone. You hear one type of transducer sounds better than another. Why is that? Well, you have to figure out some way to see it (literally), via a scope, a charting pen, something that tells you in an objective way why they are different, that allows you to set a standard or goal and move towards it.
This person probably did not set out to measure all possible things. Maybe the first thing they decide to measure is distortion, or perhaps frequency response. After visualizing the raw data the scientist then has to decide what the units are, and how to express differences. Lets say it is distortion. In theory, there could have been a lot of different ways to measure distortion. Such as Vrms - Vrms (expected) /Hz. Depending on the engineer's need at the time, that might have been a perfectly valid way to measure the output.
But here's the issue. This may work for this engineer solving this time, and we may even add it to the cannon of common measurements, but we are by no means done.
So, when exactly are we done?? At 1? 2? 5? 30? The answer is we are not. There are several common measurements for speakers for instance which I believe should be done more by reviewers:
- Compression
- Intermodulation ( IM ) Distortion
- Distortion
and yet, we do not. IM distortion is kind of interesting because I had heard about it before from M&K's literature, but it reappeared for me in the blog of Roger Russel ( http://www.roger-russell.com ) formerly from McIntosh. I can't find the blog post, but apparently they used IM distortion measurements to compare the audibility of woofer changes quite successfully.
Here's a great example of a new measurement being used and attributed to a sonic characteristic. Imagine the before and after. Before using IM, maybe only distortion would have been used. They were of course measuring impedance and frequency response, and simple harmonic distortion, but Roger and his partner could hear something different not expressed in these measurements, so, they invent the use of it here. That invention is, in my mind, actual audio science.
The opposite of science would have been to say "frequency, impedance, and distortion" are the 3 characteristics which are audible, forever. Nelson pass working with the distortion profile, comparing the audible results and saying "this is an important feature" is also science. He's throwing out the normal distortion ratings and creating a whole new set of target behavior based on his experiments. Given the market acceptance of his very expensive products I'd say he's been damn good at this.
What is my point to all of this? Measurements in the consumer literature have become complacent. We've become far too willing to accept the limits of measurements from the 1980's and fail to develop new standard ways of testing. As a result of this we have devolved into camps who say that 1980's measures are all we need, those who eschew measurements and very little being done to show us new ways of looking at complex behaviors. Some areas where I believe measurements should be improved:
We have become far too happy with this stale condition, and, for the consumers, science is dead.
"If you can't measure it with common, existing measurements it isn't real."
This idea is and always will be flawed. Mind you, maybe what you perceive is not worth $1, but this is not how science works. I'm reminded of how many doctors and scientists fought against modernizing polio interventions, and how only recently did the treatment for stomach ulcers change radically due to the curiosity of a pair of forensic scientists.
Perception precedes measurement. In between perception and measurement is (always) transference to visual data. Lets take an example.
You are working on phone technology shortly after Bell invents the telephone. You hear one type of transducer sounds better than another. Why is that? Well, you have to figure out some way to see it (literally), via a scope, a charting pen, something that tells you in an objective way why they are different, that allows you to set a standard or goal and move towards it.
This person probably did not set out to measure all possible things. Maybe the first thing they decide to measure is distortion, or perhaps frequency response. After visualizing the raw data the scientist then has to decide what the units are, and how to express differences. Lets say it is distortion. In theory, there could have been a lot of different ways to measure distortion. Such as Vrms - Vrms (expected) /Hz. Depending on the engineer's need at the time, that might have been a perfectly valid way to measure the output.
But here's the issue. This may work for this engineer solving this time, and we may even add it to the cannon of common measurements, but we are by no means done.
So, when exactly are we done?? At 1? 2? 5? 30? The answer is we are not. There are several common measurements for speakers for instance which I believe should be done more by reviewers:
- Compression
- Intermodulation ( IM ) Distortion
- Distortion
and yet, we do not. IM distortion is kind of interesting because I had heard about it before from M&K's literature, but it reappeared for me in the blog of Roger Russel ( http://www.roger-russell.com ) formerly from McIntosh. I can't find the blog post, but apparently they used IM distortion measurements to compare the audibility of woofer changes quite successfully.
Here's a great example of a new measurement being used and attributed to a sonic characteristic. Imagine the before and after. Before using IM, maybe only distortion would have been used. They were of course measuring impedance and frequency response, and simple harmonic distortion, but Roger and his partner could hear something different not expressed in these measurements, so, they invent the use of it here. That invention is, in my mind, actual audio science.
The opposite of science would have been to say "frequency, impedance, and distortion" are the 3 characteristics which are audible, forever. Nelson pass working with the distortion profile, comparing the audible results and saying "this is an important feature" is also science. He's throwing out the normal distortion ratings and creating a whole new set of target behavior based on his experiments. Given the market acceptance of his very expensive products I'd say he's been damn good at this.
What is my point to all of this? Measurements in the consumer literature have become complacent. We've become far too willing to accept the limits of measurements from the 1980's and fail to develop new standard ways of testing. As a result of this we have devolved into camps who say that 1980's measures are all we need, those who eschew measurements and very little being done to show us new ways of looking at complex behaviors. Some areas where I believe measurements should be improved:
- The effects of vibration on ss equipment
- Capacitor technology
- Interaction of linear amps with cables and speaker impedance.
We have become far too happy with this stale condition, and, for the consumers, science is dead.
335 responses Add your response
Very interesting thread thanks.... Perception precedes measurement. In between perception and measurement is (always) transference to visual dataI understand the motivation of your inquiry....Very good questions... I will only add that what the human ears think of and create of his own, cannot be totally exhausted by any actual or future measurements....Anymore than life can be replicated in laboratory.... But your question is an appeal to a more creative path to innovate new ways in audio and I think you articulate very well your points and questioning ….I will read with pleasure and curiosity all the answers of more talented and knowledgeable people about it than me here.... Thanks.... |
erik_squires
We have become far too happy with this stale condition, and, for the consumers, science is dead.This is the logical fallacy of the excluded middle. That measurements may be imperfect does not support the conclusion that science is dead for the consumer. Far from it: Science and technology march on, and the consumer benefits. Yes, measurements can be improved, no doubt. Have at it @erik squires, and good luck! |
Erik wrote: "Measurements in the consumer literature have become complacent. We’ve become far too willing to accept the limits of measurements from the 1980’s and fail to develop new standard ways of testing. As a result of this we have devolved into camps who say that 1980’s measures are all we need, those who eschew measurements and very little being done to show us new ways of looking at complex behaviors." Total harmonic distortion is not a useful metric, as it has been shown to have a slight negative correlation to listener preference under controlled conditions. Human hearing is complex and what actually matters to the ear is often counter-intuitive. If we are going to use "distortion" as a metric of merit, we should process our measurements in such a way that the result correlates well with human perception. A metric which is demonstrably far more predictive of perception than THD has been figured out. It’s called the GedLee Metric, but it has not yet gained widespread acceptance. Duke |
Duke: @audiokinesis Don’t focus on distortion. Focus on us in consumer land (albeit educated consumer) having a limited number of measures and assuming we can’t have or use more. Here I don't care about distortion, I used it as an example of how we invent specifications and measurements. I could have used gain instead for the example. Best, E |
Focus on us in consumer land (albeit educated consumer) having a limited number of measures and assuming we can’t have or use more. Why didn't you say so in the first place then. So now the question is, why would we want more measurements? I mean, you just showed them to be arbitrary, and Duke showed at least one to actually be leading the wrong way. So why more measurements? Seems to me all measurements do is create the illusion of objectivity, when in fact they are anything but. The very act of choosing which aspect to measure is subjective. The means of measurement, unit of measurement, and circumstances, these are all value judgments. Even the decision whether or not to consider double-blind testing appropriate or valuable is itself a subjective value judgment. One after another every single measurement boils down to being subjective, on top of inadequate. Why would anyone want even more of that? I mean, I can understand a designer. Some guy like Duke or anyone trying to design something better, they need something to focus on to test their theory out on. So they do need to measure. But you didn't ask about that. You specifically target consumers. Seems to me the consumer will be a lot better off to accept the fundamental enduring inadequate nature of measurement and focus all their time and energy on the one thing that really does work and that they do have control over, and that is listening. |
I never said measurements were arbitrary: https://www.merriam-webster.com/dictionary/arbitrary I said they are created by necessity to meet a specific need, as opposed to measuring everything humans can hear in all cases. To reiterate: What is my point to all of this? Measurements in the consumer literature have become complacent. |
Erik wrote: "Don’t focus on distortion. Focus on us in consumer land (albeit educated consumer) having a limited number of measures and assuming we can’t have or use more." I am confused. Can you restate what you are looking for? I’m sure it’s clear to you, but I am not getting it. Are you looking for a plain-English translation of what a designer focuses on? Duke |
@audiokinesis The point was that we talk about a handful of measurements as if they were all we needed and all we should ever have, and we call this science. It isn’t. It’s not science unless we are getting better and better at inventing measurements which reflect human perception. We cannot know from an (used correctly) arbitrary count of measurements everything about how we perceive it. Imagine if as Bell’s assistant, I said "amplitude at 1 kHz is everything and all humans can perceive." Well, clearly that’s not what we consider adequate, but it might be enough at this time in history when I’m trying to pick among 4 different diaphragm types. The types of measurements we are using was not my complaint, it’s that we are happy with what we have. From Bell's time to the 1970s/ 1980s the popular literature and the hobbyist expanded what we think of as the basic measurement suite. When and why did it stop? Best, E |
| Post removed |
audiokinesis "If we are going to use "distortion" as a metric of merit, we should process our measurements in such a way that the result correlates well with human perception." In other words more precise and appropriate data - not less. Remember what the OP said? "Measurements in the consumer literature have become complacent. We've become far too willing to accept the limits of measurements from the 1980's and fail to develop new standard ways of testing." The idiotic Hi-Fi press is a laughing stock nowadays. Just check out any car magazine and you will see reams of data pouring off the pages. Everything from cabin and tyre noise to the usual 0-60, fuel consumption, BHP/torque graphs etc. Why is the audio consumer treated with such comparable contempt? |
| Post removed |
What specification would tell you that most amps sound better driving an easy load than they do putting out the additional power? This at least has some measurements, Steroephile routinely publishes power vs. load and frequency response vs. a simulated speaker load. But we're not talking about what the manufacturers may feel obligated to publish, per se. I want to see science, as we the educated consumers know of it, to progress. BTW, there's really interesting stuff about what makes "science" science if any of you ever read subjects on the history of science. Sometimes science is defined by the consumers as much as by the practitioners. Phrenology was science to many. We can laugh at it now, but at the time, without the benefit of hindsight that was real. |
Erik, I don’t think it is so much the death of science that has stopped the adoption of better measurement methods so much as it has been the rise of marketing. Not just the sheer amounts of advertising, but journalism and also the advent of the internet of plenty have conspired to distract us (the consumers) from the notion or concept of what a relevant set of measurements might look like or the significance of its role. In the 80’s the entire hobby was accessed through b&m stores and magazines. It was a lot easier for most everyone to point to an article now and again that might propose or explore new test techniques, since that put it still within the bounds of usual discussion. But, since then the sheer amount of information that has been made available to us average consumers has mushroomed. Combined with the internet, and suddenly ’everyone is an expert’ in this information glut. And an informed/distracted (from the original idea of a standardizing of test methodology, in this case) public, in that regard, will almost certainly be marketed to differently. Once that notion fell out of public consciousness, then the pressure was off of manufacturers or advertisers to maintain it. But, dial back the clock on any audio discussion and you might find people talking about at least Some issues that...well, maybe..we might ought to be still talking about even today. Time and innovation march on, but, in our haste to get to the future sometimes, it seems there may not be anyone in charge of making sure that we remember to bring along Everything that might be important...or that might prove to be. Manufacturers might listen though - IF everyone were telling them the same thing. But, which comes first, the manufacturers talking about the right specs, or the consumers...the chicken or the egg?? |
Erik, I agree with the premise that we need more informative measurements (or, really, more informative measures, the difference being that the latter are derived from the former, using some formula that relates to perception). I am, however, puzzled by your seeming dismissal of the one thing mentioned here that seems to represent progress along those lines, the GedLee Metric mentioned by @audiokinesis. |
I am, however, puzzled by your seeming dismissal of the one thing mentioned here that seems to represent progress along those lines, the GedLee Metric mentioned by @audiokinesis. I didn't dismiss it, I also did not address it. He wrote: A metric which is demonstrably far more predictive of perception than THD has been figured out. It’s called the GedLee Metric, but it has not yet gained widespread acceptance. I know nothing about it, so I wasn't sure if he was being facetious or not. And it's the lack of widespread knowledge of these metrics which hinders us. Like, how many knew IM distortion was ever measured for speakers? |
OK, I’ve briefly read the AES papers. So Duke proposes a way of modelling systems so that multiple non-linear behaviors can be amalgamated at once, and then comparing that to what is known about auditory masking perception, further he’s actually tested this out with different populations of listeners? Outsanding. :) Pretty ambitious too. The idea of modelling multiple non-linear systems at once to derive a master model of behavior could probably be it’s own thing, but hey, I've never published anything. |
As others have pointed out, correctly, science actually IS continuing. The GedLee metric is in fact an example of what we need. The barrier, IMHO is us the merely educated consumer. There may be a thousand new interesting metrics proposed via the AES or used in secret, but so long as we are limited in what comes down the journalism pipeline we’ll not really get better. If I may, we can use the GM metric to also demonstrate the fallacy we see here in our hobby circle: The idea that all is known by existing/old metrics. GM makes the clear case that for specific uses, it does not. Kind of blows up the "if you can't measure it with moldy old metrics, you can't hear it" argument. |
Erik: "OK, I’ve briefly read the AES papers [pertaining to the GedLee Metric]. So Duke proposes a way of modelling systems so that multiple non-linear behaviors can be amalgamated at once, and then comparing that to what is known about auditory masking perception..." For the record, Duke had nothing to do with the development of the GedLee Metric. It was entirely the work of Dr. Earl Geddes and Dr. Lydia Lee. Erik: "... further he’s actually tested this out with different populations of listeners?" The papers were peer reviewed, which implies that the test methodology and its subsequent analysis stood up to critique by those with expertise in all of the relevant areas, along with a duty to uphold the credibility of the Audio Engineering Society. Erik: "... science actually IS continuing. The GedLee metric is in fact an example of what we need. The barrier, IMHO is us the merely educated consumer. " Thank you for taking a look at it. Briefly, the GedLee Metric examines the transfer function of a device (how that device alters the input signal) through a psychoacoustic (i.e. perception-based) lens. Erik: "The barrier, IMHO is us the merely educated consumer." You sir have done a great deal on this site to educate consumers. And educated audiophiles push the industry to evolve by financially rewarding companies which are genuinely advancing the art. Thank you! Duke |
"...Seems to me the consumer will be a lot better off to accept the fundamental enduring inadequate nature of measurement and focus all their time and energy on the one thing that really does work and that they do have control over, and that is listening." This should not be controversial (or at least the 2nd part shouldn’t be). |
I recall Einstein's thought experiments ... before his time, it's kind of hard to find equipment like oscilloscopes, DMM, atom smasher and stuffs. All he had was his mind and his own thought experiments. Now with the advent of new technology, people now are looking down at it like some kind of taboo. |
I would merely suggest that neurology is the area of science that presents the greatest immediacy and complexity. The question of measurements of physical phenomena has merit and relevance. But all perception goes through the labyrinth and mutability of the brain and its neurological extension. Science is not limited to the measurement of apparently physical phenomena. |
Yes, there are measurements and they are valid, but nobody that I’ve known of could figure out how to measure our "hearing". For example, there is no measurement I’ve known of that can tell how good a woofer is by just looking at the freq. and phase plots. MC is a controversial figure, but he said something I would agree. He said that Mercedes have spent millions of dollars to design all sort of sensors but at the end they have to rely on Lewis Hamilton to tell them what’s going on with the car. This does not mean all the measurements made by Mercedes was not valid, it’s just that there is a human element that cannot be measured. Same with speaker design. After all the simulations, fine tuning the freq. and phase plot, one still have to sit down and listen and judge with your own ears. One cannot judge a pair of speaker with the freq and phase plot. I don’t mean to disregard measurements. On the other hands, quite the opposite. All the improvements in cables, drivers, capacitors, inductors would not have happened without the advance in measurement equipment and software. I find it amusing that people are pitting the "measurement" vs. "hearing" as some type of a fight. It's like saying which is better - apples or oranges. |
As you narrow down the scope of the problem to some specific variables, then measurements can be used to quantify the "goodness" of that something. For example, it may be hard to judge how good a pair of speaker with measurement, but if the device under test is a single capacitor or inductor, then it's easier to come up with a set of measurement to quantify the performance of said component. But then at the end, I am afraid one still has to listen - God forbids. |
I recall Einstein's thought experiments ... before his time, it's kind of hard to find equipment like oscilloscopes, DMM, atom smasher and stuffs. True, Einstein theorized many things, but .... since then scientists have been in a rush to confirm or refute many of them. Einstein was also sometimes wrong. |
Andy2 wrote: "One cannot judge a pair of speaker with the freq and phase plot." Agreed, a frequency plot and a phase plot are insufficient. But there is fairly good evidence that one can judge a pair of speakers with a sufficiently comprehensive suite of the right measurements. This is the guy who wrote the book on the subject: https://youtu.be/zrpUDuUtxPM?t=259 (Up to about 4:20 is just introduction.) And here is the book: https://www.amazon.com/Sound-Reproduction-Psychoacoustics-Loudspeakers-Engineering/dp/113892136X/ref... (I recommend the Third Edition; the writing style is a bit easier read for the non-technical audiophile, and the author includes personal experiences which are imo very informative.) Duke |
Erik says “The idea of modeling multiple non-linear systems at once to derive a master model of behavior could probably be its’ own thing.” It certainly is. I recently finished a little project where I applied a machine-learning neural network model to classify 1000 clips of music according to a ‘harmonic signature’ (mostly “live content” case #1, or mostly “synthesized”, case #2). Interestingly, although FFT, spectral centroids, RMS energy etc. were important in defining the signature, the most compelling predictors were the Mel-Frequency Cepstral Coefficients. Why is this important? Because the purpose of these coefficients is try to capture exactly what audiokineses refers to as “the transfer function of a device (how that device alters the input signal) through a psychoacoustic (i.e. perception-based) lens.” Two examples will explain: ‘the sound of a baby crying.’ Why is the baby crying? Hungry? Lonely? Needs diaper changed? We could do technical analysis forever and not know – but the mother knows instantly. Someone singing, “I don’t know what to do.” Why? Boredom? Lost love? We know from the voice; not just the context. I think that psychoacoustic perception is exactly where we need to look to understand that last 10% or 20% beyond the point where purely technical/engineering analysis stops reliably explaining what we know to be true in our ears. The science is not dead; it’s getting more and more interesting. |
"Once we get into the non-linear things, it can get complicated real fast." - Richard Feynman Very true!! Fortunately as long as loudspeakers are not pushed past their linear excursion limits, non-linear distortion is highly unlikely. What is NOT unlikely is this: Linear distortions to which the EAR has a NON-LINEAR perception response. So it can sound like a system distorts as the SPL goes up, when what is really happening is, the ear is having a non-linear perception of what is actually a linear distortion. This is one of the reasons why listening tests should include some high-SPL listening, because there are linear distortions which are inaudible at low SPLs but which can set your teeth on edge at high SPLs. Duke |
So what do room measurements and treatments (or old fashioned equalizers) bring to the table? Does it not take flawed sound, badly measured sound if you will, and adjust it to measure better for the individual consumer in a specific locale. This would seem a better solution as it incorporates unique parameters; room, my equipment, and my ears. I understand the desire to eliminate sow's ears but you assume objective, standardized, and constantly improving measurement techniques by audio press or manufacturers. Not sure about that. Joe |
ivan_nosnibor "Erik, I don’t think it is so much the death of science that has stopped the adoption of better measurement methods so much as it has been the rise of marketing. Not just the sheer amounts of advertising, but journalism and also the advent of the internet of plenty have conspired to distract us (the consumers) from the notion or concept of what a relevant set of measurements might look like or the significance of its role." Nailed it! The entire industry currently runs on implicit advertising and marketing. It discovered decades ago that honesty and openness have little or no relationship to commercial success. There are no longer any actual claims of sonic superiority in any audio product now being marketed. None whatsoever. Think about that. And more to the point, why should they bother when we golden-eared consumers can't even agree whether we want more data, or even which data we particularly want? Children in the world of adults. |
| Post removed |
It is the human ears that judge the superiority of any electronic component, not in the abstract, disembodied standardized field of measurement necessary for implementation of the engineering protocols, but in the embodied particular multiple embeddings of your room and house and in a particular individualized audio system.... Then the most important underestimated facts in audio are the controls of the 4 basic embeddings : mechanical, electrical, and the passive and active controls of the acoustical field of the room ... There are others dimensions but these 4 one are fundamentals... Buying first a high quality design electronic component is very important, but the evolution of the design of electronic component in the last 50 years, as big as it is, is not on par, nor on the same scale, than the difference in S.Q. gained by a rightful implementation of controls protocols for the 4 embeddings... This is my audiophile journey lesson...Nobody explain me that clearly, I discover it myself by listening experiments, slowly first in the first 5 years, and swiftly in the last 2 years... Improving electronic design is very important, but learning to listen and resolving in a simple and affordable manner the 4 embeddings problems are the crux of audio and the more fundamental problem... Most of us we owns already a good audio system, the real question is not about the way to upgrade the design of an electronic component by buying a new improved one; the real question is : Do I know from having heard it already, how my audio system, as it is now, behaves in an optimal controlled environment ? The answer for most people will be no....Frustrated by the limitations of their actual system, without knowing that the limitations comes from the lack of control of the embeddings, more than from the already good design of their actual system " per se", they turn themselves prey to marketing ploy and upgrade, without even having heard their system in his optimal possibilities to begins with... This is what I learned ... Note: I am not a closed mind, and I am conscious of the importance of improved engineering methods and products... I looked about that suggested here and it seems interesting : the GedLee Metric.. I am interested for sure also in new design, for example ZOTL technology... |
@alymere The science is not dead; it’s getting more and more interesting.I agree. I think the general consensus here is (& @erik_squires certainly makes the case): we measure irrelevant things. So, it's not science but how it is applied to audio that is (mostly) dead (of meaning). Mostly... (BTW, speaker response measurements, can be very useful; MLSSA & FFT are very useful tools. ) |
How can the audio industry (HIFI) benefit from new measurements that would shine a light on whats really going on? Audiophiles are paying for what subjectively steals headlines. Look at how the Wilson Sasha daw measures in room and tell me consumers want more measurements? sorry, I agree with you but telling folk that spend as much on wire as we do on the family car they need more measurements is going to bounce. just sayin'. |
| Post removed |
