One topic that often comes up is perception vs. measurements.
"If you can't measure it with common, existing measurements it isn't real."
This idea is and always will be flawed. Mind you, maybe what you perceive is not worth $1, but this is not how science works. I'm reminded of how many doctors and scientists fought against modernizing polio interventions, and how only recently did the treatment for stomach ulcers change radically due to the curiosity of a pair of forensic scientists.
Perception precedes measurement. In between perception and measurement is (always) transference to visual data. Lets take an example.
You are working on phone technology shortly after Bell invents the telephone. You hear one type of transducer sounds better than another. Why is that? Well, you have to figure out some way to see it (literally), via a scope, a charting pen, something that tells you in an objective way why they are different, that allows you to set a standard or goal and move towards it.
This person probably did not set out to measure all possible things. Maybe the first thing they decide to measure is distortion, or perhaps frequency response. After visualizing the raw data the scientist then has to decide what the units are, and how to express differences. Lets say it is distortion. In theory, there could have been a lot of different ways to measure distortion. Such as Vrms - Vrms (expected) /Hz. Depending on the engineer's need at the time, that might have been a perfectly valid way to measure the output.
But here's the issue. This may work for this engineer solving this time, and we may even add it to the cannon of common measurements, but we are by no means done.
So, when exactly are we done?? At 1? 2? 5? 30? The answer is we are not. There are several common measurements for speakers for instance which I believe should be done more by reviewers:
- Compression - Intermodulation ( IM ) Distortion - Distortion
and yet, we do not. IM distortion is kind of interesting because I had heard about it before from M&K's literature, but it reappeared for me in the blog of Roger Russel ( http://www.roger-russell.com ) formerly from McIntosh. I can't find the blog post, but apparently they used IM distortion measurements to compare the audibility of woofer changes quite successfully.
Here's a great example of a new measurement being used and attributed to a sonic characteristic. Imagine the before and after. Before using IM, maybe only distortion would have been used. They were of course measuring impedance and frequency response, and simple harmonic distortion, but Roger and his partner could hear something different not expressed in these measurements, so, they invent the use of it here. That invention is, in my mind, actual audio science.
The opposite of science would have been to say "frequency, impedance, and distortion" are the 3 characteristics which are audible, forever. Nelson pass working with the distortion profile, comparing the audible results and saying "this is an important feature" is also science. He's throwing out the normal distortion ratings and creating a whole new set of target behavior based on his experiments. Given the market acceptance of his very expensive products I'd say he's been damn good at this.
What is my point to all of this? Measurements in the consumer literature have become complacent. We've become far too willing to accept the limits of measurements from the 1980's and fail to develop new standard ways of testing. As a result of this we have devolved into camps who say that 1980's measures are all we need, those who eschew measurements and very little being done to show us new ways of looking at complex behaviors. Some areas where I believe measurements should be improved:
The effects of vibration on ss equipment
Capacitor technology
Interaction of linear amps with cables and speaker impedance.
We have become far too happy with this stale condition, and, for the consumers, science is dead.
Andy2 wrote: "One cannot judge a pair of speaker with the freq and phase plot."
Agreed, a frequency plot and a phase plot are insufficient.
But there is fairly good evidence that one can judge a pair of speakers with a sufficiently comprehensive suite of the right measurements. This is the guy who wrote the book on the subject:
Death of science? What does that mean? Buyers aren’t scientists. Also there are new innovations all the time....thanks to science. Amazingly the good ones catch on.
I recall Einstein's thought experiments ... before his time, it's kind
of hard to find equipment like oscilloscopes, DMM, atom smasher and
stuffs.
True, Einstein theorized many things, but .... since then scientists have been in a rush to confirm or refute many of them. Einstein was also sometimes wrong.
As you narrow down the scope of the problem to some specific variables, then measurements can be used to quantify the "goodness" of that something. For example, it may be hard to judge how good a pair of speaker with measurement, but if the device under test is a single capacitor or inductor, then it's easier to come up with a set of measurement to quantify the performance of said component.
But then at the end, I am afraid one still has to listen - God forbids.
Yes, there are measurements and they are valid, but nobody that I’ve known of could figure out how to measure our "hearing".
For example, there is no measurement I’ve known of that can tell how good a woofer is by just looking at the freq. and phase plots.
MC is a controversial figure, but he said something I would agree. He said that Mercedes have spent millions of dollars to design all sort of sensors but at the end they have to rely on Lewis Hamilton to tell them what’s going on with the car. This does not mean all the measurements made by Mercedes was not valid, it’s just that there is a human element that cannot be measured.
Same with speaker design. After all the simulations, fine tuning the freq. and phase plot, one still have to sit down and listen and judge with your own ears. One cannot judge a pair of speaker with the freq and phase plot.
I don’t mean to disregard measurements. On the other hands, quite the opposite. All the improvements in cables, drivers, capacitors, inductors would not have happened without the advance in measurement equipment and software.
I find it amusing that people are pitting the "measurement" vs. "hearing" as some type of a fight. It's like saying which is better - apples or oranges.
I would merely suggest that neurology is the area of science that presents the greatest immediacy and complexity. The question of measurements of physical phenomena has merit and relevance. But all perception goes through the labyrinth and mutability of the brain and its neurological extension. Science is not limited to the measurement of apparently physical phenomena.
I recall Einstein's thought experiments ... before his time, it's kind of hard to find equipment like oscilloscopes, DMM, atom smasher and stuffs. All he had was his mind and his own thought experiments. Now with the advent of new technology, people now are looking down at it like some kind of taboo.
"...Seems to me the consumer will be a lot better off to accept the fundamental enduring inadequate nature of measurement and focus all their time and energy on the one thing that really does work and that they do have control over, and that is listening."
This should not be controversial (or at least the 2nd part shouldn’t be).
Erik: "OK, I’ve briefly read the AES papers [pertaining to the GedLee Metric]. So Duke proposes a way of modelling systems so that multiple non-linear behaviors can be amalgamated at once, and then comparing that to what is known about auditory masking perception..."
For the record, Duke had nothing to do with the development of the GedLee Metric. It was entirely the work of Dr. Earl Geddes and Dr. Lydia Lee.
Erik: "... further he’s actually tested this out with different populations of listeners?"
The papers were peer reviewed, which implies that the test methodology and its subsequent analysis stood up to critique by those with expertise in all of the relevant areas, along with a duty to uphold the credibility of the Audio Engineering Society.
Erik: "... science actually IS continuing. The GedLee metric is in fact an example of what we need. The barrier, IMHO is us the merely educated consumer. "
Thank you for taking a look at it.
Briefly, the GedLee Metric examines the transfer function of a device (how that device alters the input signal) through a psychoacoustic (i.e. perception-based) lens.
Erik: "The barrier, IMHO is us the merely educated consumer."
You sir have done a great deal on this site to educate consumers. And educated audiophiles push the industry to evolve by financially rewarding companies which are genuinely advancing the art.
As others have pointed out, correctly, science actually IS continuing. The GedLee metric is in fact an example of what we need. The barrier, IMHO is us the merely educated consumer. There may be a thousand new interesting metrics proposed via the AES or used in secret, but so long as we are limited in what comes down the journalism pipeline we’ll not really get better.
If I may, we can use the GM metric to also demonstrate the fallacy we see here in our hobby circle: The idea that all is known by existing/old metrics. GM makes the clear case that for specific uses, it does not. Kind of blows up the "if you can't measure it with moldy old metrics, you can't hear it" argument.
OK, I’ve briefly read the AES papers. So Duke proposes a way of modelling systems so that multiple non-linear behaviors can be amalgamated at once, and then comparing that to what is known about auditory masking perception, further he’s actually tested this out with different populations of listeners? Outsanding. :)
Pretty ambitious too. The idea of modelling multiple non-linear systems at once to derive a master model of behavior could probably be it’s own thing, but hey, I've never published anything.
I am, however, puzzled by your seeming dismissal of the one thing
mentioned here that seems to represent progress along those lines, the
GedLee Metric mentioned by @audiokinesis.
I didn't dismiss it, I also did not address it. He wrote:
A metric which is demonstrably far more predictive of perception than
THD has been figured out. It’s called the GedLee Metric, but it has not
yet gained widespread acceptance.
I know nothing about it, so I wasn't sure if he was being facetious or not. And it's the lack of widespread knowledge of these metrics which hinders us.
Like, how many knew IM distortion was ever measured for speakers?
Erik, I agree with the premise that we need more informative measurements (or, really, more informative measures, the difference being that the latter are derived from the former, using some formula that relates to perception). I am, however, puzzled by your seeming dismissal of the one thing mentioned here that seems to represent progress along those lines, the GedLee Metric mentioned by @audiokinesis.
Erik, I don’t think it is so much the death of science that has stopped the adoption of better measurement methods so much as it has been the rise of marketing. Not just the sheer amounts of advertising, but journalism and also the advent of the internet of plenty have conspired to distract us (the consumers) from the notion or concept of what a relevant set of measurements might look like or the significance of its role. In the 80’s the entire hobby was accessed through b&m stores and magazines. It was a lot easier for most everyone to point to an article now and again that might propose or explore new test techniques, since that put it still within the bounds of usual discussion. But, since then the sheer amount of information that has been made available to us average consumers has mushroomed. Combined with the internet, and suddenly ’everyone is an expert’ in this information glut. And an informed/distracted (from the original idea of a standardizing of test methodology, in this case) public, in that regard, will almost certainly be marketed to differently. Once that notion fell out of public consciousness, then the pressure was off of manufacturers or advertisers to maintain it.
But, dial back the clock on any audio discussion and you might find people talking about at least Some issues that...well, maybe..we might ought to be still talking about even today. Time and innovation march on, but, in our haste to get to the future sometimes, it seems there may not be anyone in charge of making sure that we remember to bring along Everything that might be important...or that might prove to be.
Manufacturers might listen though - IF everyone were telling them the same thing. But, which comes first, the manufacturers talking about the right specs, or the consumers...the chicken or the egg??
What specification would tell you that most amps sound better driving an
easy load than they do putting out the additional power?
This at least has some measurements, Steroephile routinely publishes power vs. load and frequency response vs. a simulated speaker load.
But we're not talking about what the manufacturers may feel obligated to publish, per se. I want to see science, as we the educated consumers know of it, to progress.
BTW, there's really interesting stuff about what makes "science" science if any of you ever read subjects on the history of science. Sometimes science is defined by the consumers as much as by the practitioners.
Phrenology was science to many. We can laugh at it now, but at the time, without the benefit of hindsight that was real.
I tihink it was likely specifications and measurements that are responsible for lower impedance speakers. What specification would tell you that most amps sound better driving an easy load than they do putting out the additional power?
@erik_squires wrote: "The point was that we talk about a handful of measurements as if they
were all we needed and all we should ever have, and we call this
science."
"If we are going to use "distortion" as a metric of merit, we should process our measurements in such a way that the result correlates well with human perception."
In other words more precise and appropriate data - not less.
Remember what the OP said?
"Measurements in the consumer literature have become complacent. We've become far too willing to accept the limits of measurements from the 1980's and fail to develop new standard ways of testing."
The idiotic Hi-Fi press is a laughing stock nowadays. Just check out any car magazine and you will see reams of data pouring off the pages. Everything from cabin and tyre noise to the usual 0-60, fuel consumption, BHP/torque graphs etc.
Why is the audio consumer treated with such comparable contempt?
With all the advancements in compute, storage and data collection technology you'd think we could see many more dynamic tests being done, but in fact they are quite rare.
The point was that we talk about a handful of measurements as if they were all we needed and all we should ever have, and we call this science.
It isn’t. It’s not science unless we are getting better and better at inventing measurements which reflect human perception. We cannot know from an (used correctly) arbitrary count of measurements everything about how we perceive it.
Imagine if as Bell’s assistant, I said "amplitude at 1 kHz is everything and all humans can perceive."
Well, clearly that’s not what we consider adequate, but it might be enough at this time in history when I’m trying to pick among 4 different diaphragm types.
The types of measurements we are using was not my complaint, it’s that we are happy with what we have. From Bell's time to the 1970s/ 1980s the popular literature and the hobbyist expanded what we think of as the basic measurement suite. When and why did it stop?
Erik wrote: "Don’t focus on distortion. Focus on us in consumer land (albeit educated consumer) having a limited number of measures and assuming we can’t have or use more."
I am confused.
Can you restate what you are looking for? I’m sure it’s clear to you, but I am not getting it. Are you looking for a plain-English translation of what a designer focuses on?
Focus on us in consumer land (albeit educated consumer) having a limited number of measures and assuming we can’t have or use more.
Why didn't you say so in the first place then. So now the question is, why would we want more measurements?
I mean, you just showed them to be arbitrary, and Duke showed at least one to actually be leading the wrong way. So why more measurements?
Seems to me all measurements do is create the illusion of objectivity, when in fact they are anything but. The very act of choosing which aspect to measure is subjective. The means of measurement, unit of measurement, and circumstances, these are all value judgments. Even the decision whether or not to consider double-blind testing appropriate or valuable is itself a subjective value judgment. One after another every single measurement boils down to being subjective, on top of inadequate. Why would anyone want even more of that?
I mean, I can understand a designer. Some guy like Duke or anyone trying to design something better, they need something to focus on to test their theory out on. So they do need to measure. But you didn't ask about that. You specifically target consumers. Seems to me the consumer will be a lot better off to accept the fundamental enduring inadequate nature of measurement and focus all their time and energy on the one thing that really does work and that they do have control over, and that is listening.
A metric which is demonstrably far more predictive of perception than THD has been figured out. It's called the GedLee Metric, but it has not yet gained widespread acceptance.
Don’t focus on distortion. Focus on us in consumer land (albeit educated consumer) having a limited number of measures and assuming we can’t have or use more.
Here I don't care about distortion, I used it as an example of how we invent specifications and measurements. I could have used gain instead for the example.
Erik wrote: "Measurements in the consumer literature have become complacent. We’ve become far too willing to accept the limits of measurements from the 1980’s and fail to develop new standard ways of testing. As a result of this we have devolved into camps who say that 1980’s measures are all we need, those who eschew measurements and very little being done to show us new ways of looking at complex behaviors."
Total harmonic distortion is not a useful metric, as it has been shown to have a slight negative correlation to listener preference under controlled conditions. Human hearing is complex and what actually matters to the ear is often counter-intuitive. If we are going to use "distortion" as a metric of merit, we should process our measurements in such a way that the result correlates well with human perception.
A metric which is demonstrably far more predictive of perception than THD has been figured out. It’s called the GedLee Metric, but it has not yet gained widespread acceptance.
We have become far too happy with this stale condition, and, for the consumers, science is dead.
This is the logical fallacy of the excluded middle. That measurements may be imperfect does not support the conclusion that science is dead for the consumer. Far from it: Science and technology march on, and the consumer benefits.
Yes, measurements can be improved, no doubt. Have at it @erik squires, and good luck!
Perception precedes measurement. In between perception and measurement is (always) transference to visual data
I understand the motivation of your inquiry....Very good questions...
I will only add that what the human ears think of and create of his own, cannot be totally exhausted by any actual or future measurements....Anymore than life can be replicated in laboratory....
But your question is an appeal to a more creative path to innovate new ways in audio and I think you articulate very well your points and questioning ….I will read with pleasure and curiosity all the answers of more talented and knowledgeable people about it than me here.... Thanks....
You must have a verified phone number and physical address in order to post in the Audiogon Forums. Please return to Audiogon.com and complete this step. If you have any questions please contact Support.
