“You should listen before measurement, or in the absence of knowing what those measurements are. If you measure, analyze and listen, in that order, you are actually creating confirmation bias.”
Absolutely this , but in some of the more esoteric pieces of hifi its even worse because the forum has picked holes in the product even before launch. Uptone audio’s Etherregen had a 20 or 30 page thread on ASR ridiculing it even before launch with Amir being one of the main protagonists. Eventually it was measured found to do nothing and then a listening test using a DAC that was also under test for a few days. You can guess how that went followed by 30 or so more pages of whooping.
In my system there was clearly audible improvements and with the vast majority of early users. |
I am not sure what kind of science and measurements Bose are using to build their speakers, but I find most of them absolutely amazing in terms of size/price/sound ratio.
I have a Bose SoundTouch 30 in my second living room and it is not less enjoyable for everyday use (of course not for a serious listening), than my main 200K+ system. |
What other scientific endeavor accepts subjective opinion as evidence and rejects scientifically accepted DBX testing as worthless? I cant think of any truly scientific endeavour that does actually use double blind testing other than clinical research and Pharmaceuticals (which is actually my field). This is vital usage as we want the best medicines and treatments available Ive seen it used in consumer product research and marketing very heavily especially food and drink. But at the end of the day it doesn’t matter how good you design cola to taste some folks are always going to prefer Pepsi over Coke because it is a subjective experience, just like hifi is. |
Oh, yeah. Concorde, XB-70. They all crashed. Spectacularly I might add.
|
Ive seen it used in consumer product research and marketing very heavily especially food and drink. But at the end of the day it doesn’t matter how good you design cola to taste some folks are always going to prefer Pepsi over Coke because it is a subjective experience, just like hifi is. At least you know it’s a subjective choice not an objective one. If you test two amps using DBT and you always prefer amp A then you know your choice is based on how it sounds. When you finally see the two amps and amp A was a cheap Class D and amp B was a high priced Class A and you buy amp B then you know your choice is like choosing Coke over Pepsi, subjective opinion. |
Any fool can hear the difference between two amps, assuming a number of things regarding the system, a level playing field as it were, which it NEVER is. Is it revealing enough, are there errors in the system, is it morning, daytime or night? Things of that nature. If you don’t trust your hearing that’s a different story. When you buy a TV you pick the one with the best picture, right? You don’t ask for DBX test do you? Well, maybe you do.
|
"Oh, yeah. Concorde, XB-70. They all crashed. Spectacularly I might add. "
Concorde crashed only once. It did not keep on crashing. XB-70 was not even the same plane. Nor was Tu-144 for whch you might try to stretch to "kept crashing". Concorde did not "keep on crashing". |
stibi,
"I have a Bose SoundTouch 30 in my second living room and it is not less enjoyable for everyday use (of course not for a serious listening), than my main 200K+ system."
What is this? Extreme high praise or an act of the highest audiophile heresy?
No worries, just joking, we know what you mean. Some of their systems might lack ultimate bandwidth, dynamics and scale, but they are sure nice to relax to at the end of a long day. |
glubson....ah, hem....smart pill reminder. 💊
|
geoffkait,
....ah, hem....memory pill reminder.💊
By the way, have you ever been in a Concorde?
|
No, but I was held prisoner by the Japanese once. I was tortured if you must know but no, no, I don’t think they wanted me to talk really, I don’t think they wanted me to say anything. It was just their way of having a bit of fun the swines. Strange thing is they make such bloody good cameras.
|
|
I am not sure what kind of science and measurements Bose are using to build their speakers, but I find most of them absolutely amazing in terms of size/price/sound ratio. And you just answered your own question, and why people buy them. |
To be clear, this is all science. Try designing an amplifier, DAC, filter, etc. without incorporating serious science. You can't. Some of you are talking like things just appear on shelves for your enjoyment without any science behind it.
Make a transistor or a vacuum tube without science. Go ahead I dare you.
People dismiss science and measurements as if they don't or shouldn't exists. They do. The issue with audio equipment and measurement is that is wasn't that important to the general well being. Who cares? just do it. But, if you put enough importance behind the why science, it would be done. Right now, it just isn't that important. Of course it can be measured. It's just not that important to spend serious scientific time and expense to do it.
Getting planes to fly correctly with the advanced circuitry involved without crashing and killing people. That is important. Getting autonomous cars to operate correctly without driving off a cliff, that takes serious measurements and is important.
Put enough time and money behind finding the why and it will be done. It is all science. This is not subjective.
Many can build an amp from off the shelf parts. But, can you actually design and build the amp from scratch understanding the math and science behind it? Can you show the equations and calculations behind why you are getting the input impedance or output impedance you desired, the gain, the frequency response? can you actually calculate and show the circuit's transfer function? do you even know what that is?
The science behind the why. What effects room characteristics have on sound, different types of transformers constructions, resistors, capacitors, coils. boy oh boy. We are talking serious science.
why does this 100 wpc amp sound great and this other 100 wpc amp sound like crap? all amps are the same right? Wrong! look at the transfer function response.
All caps are the same right? Wrong!. So the question is, what do you want to measure and is that important enough to spend tons of money and invest lots of time and effort to do it? Most cases in the audio world the answer is no. That does not mean it can't be done.
Go listen to and record a band playing in a certain venue (outside or inside) good seats. What do you hear? now if the recording is done correctly (not always the case), play it back on pretty decent equipment. Tube or solid state or both. Does it sound anywhere close to what you originally heard? Probably not.
Well since most people won't be at the original venue to hear the original performance and then hear it again recorded, some audio manufacturers "tune" their equipment to "sound good". Very subjective. Not necessary accurate technologically. An amplifier is suppose to amplify the signal exactly with no alterations other than gain. But, that is typically not the case not is it? The amp adds something(s). Typically distortion. Some "sound good" others don't. so tune the amp's distortion characteristics so that it sounds better. Doesn't mean it is accurate. From a scientific view, you would never try to add characteristics to an amp. But from a sound perspective, maybe you want to.
Anyway, stop dissing science and measurements. If it was important enough to do it, it could be and would be done.
enjoy |
I'll throw in a measurement which may go the wrong way. Linearity of a DAC. The linearity is basically that as the signal level changes (and this can be sample to sample) is the output correctly matching the input? Much like the linearity of a transistor or tube.
The measurements would indicate that a more linear DAC is a better DAC, right? Well, what if a little compression actually sounds better? For instance, as the notes decay, and we are left with the acoustics of the room, it's quite possible we prefer these ambient cues to be exaggerated so we can better hear them. I think I do, though I've not confirmed this via measurements.
It is merely engineering or QA to ascribe value based purely on best linearity. It is science to compare different linearity DAC's or signals and see which is preferred by listeners.
Does this help others see the gap I'm talking about?
Best,
E
|
Erik, I own and enjoy a DAC/headphone amp by Pro-Ject Audio that has taken a unique approach to the "science vs. art," or "measurements vs. sound" conundrum. Unlike the great designers/teams you mentioned, they chose not to decide, sort of. Cue Rush "Freewill." The Pre Box S2 Digital (awful name) offers an option to choose a "Test" configuration. This is the configuration in which the DAC measures best on the O-scope, and includes "Distortion Compensation," whatever that is. But Pro-Ject acknowledges that this mode doesn't actually sound great, and that they included this "Test" option in the menu strictly for the consumption of measurement buffs (like ASR): https://www.audiosciencereview.com/forum/index.php?threads/review-and-measurements-of-pro-ject-pre-box-s2-digital.2370/
The alternative is "Best" mode, which turns off the Distortion Compensation, and possibly does more. They say despite the testing data, it just sounds better. So I suppose they really did decide, at least for themselves, and then left implementation for the end user to decide. At first I disagreed with their recommendation for both best/test mode, and their recommendation for their proprietary reconstruction filter (for non-MQA files). As so often happens over time, I tired of the "snappy" sound and eventually embraced their implementation thoughts - smoother thoughts - to my current delight. I'm not aware of other manufacturers attempting to offer both sides of the measurements/sound quality coin, so thought you might find this approach interesting. Thanks, Erik, I enjoy your posts - JG |
geoffkait, "...no, no, I don’t think they wanted me to talk really, I don’t think they wanted me to say anything." So you think all of us here are Japanese? |
Wormtosser, Did you read through that thread? The "test" mode had more distortion, when the distortion compensation was turned off some of the numbers were better than the test mode. They couldn't figure out what was going on since it looked like distortion compensation was doing the opposite of what it was suppose to do. |
Glubson, what’s up with all the racism?
|
geoffkait, "Glubson, what’s up with all the racism?" I have no idea. Virus was more interesting. |
djones,
Great observation, and yes I did see the same - measurements were opposite of expectations or claims. Maybe early product sample not sorted out yet? Got their wires, or software, or labeling crossed? QA problem at the factory? Who knows?
Regardless, the mfr attempted to solve the measurements-vs-sound conundrum by offering both choices. Even if, given the ASR findings, you believe that they failed in the attempt, I still think its interesting that they tried.
I think in the digital realm we could see more of this, just like DAC chip mfrs offering multiple filters. Even in analog, we see tube amp mfrs offering amps instantly switchable from ultra linear to SET modes.
I like having the choices.
Thanks for commenting dj - appreciate the thought and effort put into it.
JG |
Bose products sell because they have gotten them placed in stores people shop in anyway, like Best Buy, etc and because tone deaf publications like Consumer Reports say they are great. But equating Bose's science to good sound is like a chemist using his knowledge of chemistry solely as his guide for preparing dinner. |
I’d say Bose’s science is going to be more like mini-oreos. It’s junk food, but you can’t deny they spent hundreds of thousands taste testing them, and ramping up for production.
Like it or not, Bose does NOTHING without extensive measurements, and listening panels as well as quality control. They may not be producing the product you or I would buy, they are producing the products that will sell the most to the widest audience for the most money. That is effective science, as well as marketing and advertisement.
Truthfully, I know of no audio company who does as much science and R&D as Bose.
Best,
E
|
|
If memory serves, and this is a long time ago, CR also tested Allison, and gave them a very high rating, after which Roy Allison criticized their testing and evaluation methodologies.
|
erik - CR’s testing of audio equipment at that time was very much a measurement approach - using primarily frequency response as the metric for their "accuracy" rankings. But they also wrote listening impressions. Their issues with the 901’s were the larger than life and any imaginable instrument image that they sometimes projected - e.g., a piano could be wide as the room even if the room was 15’ in width. I remember when the 901’s were introduced going to listen to them - they were pretty startling (whoa, how’d that guitarist get on the ceiling?!!?) and, even taking into account the limitations of the most popular speakers at the time (which were primarily bookshelf models), they departed considerably from the norm as well as what one might expect from live music. That was the first iteration - I admit that I have not listened to the later models. Although I don’t remember the particulars, my guess is that Roy Allison objected to the FR measure that was CR’s primary accuracy criterion which, I believe, was conducted in an anechoic chamber. That, of course, would negate the feature of Allison speakers that distinguished them from most others (positioning close to the wall to take advantage of the reinforcement effects the boundary afforded.
|
If I told you Karl-Anthony Towns is just as good as LeBron James because they both averaged the same number of points per game, would you buy that? What if I went further and said and perceived greatness of LeBron was simply snake oil that you dreamed up in your head?
There has been a revolution in sports analytics in this century to the point where it is now very easy to show why LeBron was better than KAT in a number of different ways.
The audio measurements crowd needs to similarly step up its game. Distortion and frequency response is points and rebounds. Points and rebounds are important, but Karl-Anthony Towns ain't leading you to a championship. |
Good analogy whoopycat. We should continue to investigate the nature of audio reproduction, perception, and appreciation. Certainly, at this point in time all of the meaningful measures have yet to be defined and curiosity about what is missing is a primary driver for scientific investigation. The bottom line, to me at least, is that we can draw some conclusions from the measurements we can make, but we certainly can't completely describe the behavior of any piece of gear nor how we will perceive it. At this point, the quote below certainly obtains. "Not everything that counts can counted and not everything that can be counted counts." (commonly attributed to Albert Einstein, but probably coined by William Bruce Cameron).
|
whoopycat,
I'm sorry but I don't see any correlation between studying random seasonal basketball stats and studying precise electrical measurements.
None whatsoever.
|
cd, I didn't say anybody should study basketball stats, but I tell you what, let's trade. You send me the precise electrical measurements for midrange transparency, and I'll send you the NBA players who were the best this year at defending the pick & roll. Oh heck, I know you're not interested, but here they are anyway: https://stats.nba.com/players/ball-handler/#!?sort=SCORE_POSS_PCT&dir=-1&SeasonType=Regular%20Season&TypeGrouping=defensiveThe point of my analogy is if you were to ask for those stats 20 years ago you would've just gotten laughed at. Now they are at anyone's fingertips. Meanwhile in home audio, we're still looking at the same old measurements of frequency response and distortion and saying this is what something sounds like. As Erik said, that's not science. Precise or otherwise, that's the equivalent of heading to a sports analytics conference with a pack of baseball cards. |
whoopycat, I understand the emergence of sporting stats. Modern day football (soccer) is now riddled with terms like assists, blocks, tackles etc which were not deemed necessary some 20/30 years ago. I still believe there’s only a dubious relationship between sporting stats and sporting success, (as it is with any other form of social science). One that you can only attempt to establish if you highlight the ones which add credence to your argument whilst ignoring the ones that don’t. Sport is a one off event with many, many variables. You’d need a computer like the one Douglas Adams wrote about in The Hitchhiker’s Guide to the Galaxy. ["You know nothing of future time," pronounced Deep Thought, "and yet in my teeming circuitry I can navigate the infinite delta streams of future probability and see that there must one day come a computer whose merest operational parameters I am not worthy to calculate, but which it will be my fate eventually to design."] However, loudspeaker testing and analysis has come a very long way with the advent of computer software and can be deemed a true science. Nowadays everyone uses this software and testing is as rigourous as its ever been. They know what to look for: eg • On-axis frequency response • Impulse response • Cumulative spectral decay • Polar response • Step response • Impedance • Efficiency/Sensitivity • Distortion • Dynamics https://audioxpress.com/article/testing-loudspeakers-which-measurements-matter-part-1https://audioxpress.com/article/testing-loudspeakers-which-measurements-matter-part-2Heck, there’s even at least one website devoted to this kind of thing. https://www.audiosciencereview.com/forum/index.phpThe true test is whether the loudspeakers are actually getting better. By and large, I’d say they were. But I acknowledge there will always remain a subjective element. Some will always prefer cassette tape to CD, a 1960 Ferrari to one from 2020, a Technics turntable from 1970 to one from now etc. That’s human nature. Fair enough, but let’s not kid ourselves about which ones measure better. |
However, loudspeaker testing and analysis has come a very long way with the advent of computer software and can be deemed a true science. My point was never that science does not occur. My point was that judging products by an oscilloscope alone, when you have not done the work of measuring user preference, or listening for yourself, is _not_ science. It’s a mechanical process of measurement, or quality assurance at best. When you are designing a piece of equipment to go in a car, air conditioner or computer, yes, that's all you need. Set the parameters, set your tolerances, and attempt to achieve those values for the least amount of money. Judging a product for human consumption on that basis alone is where the term science is misapplied, unless you have previously worked to establish those were desirable. Best, E |
erik_squires,
"..unless you have previously worked to establish those were desirable."
Could not agree more, but isn’t that what Toole, Olive and co have been doing at Harman for the past 30 years or so?
Furthermore as some have mentioned here before, isn’t it high time the consumers were given all the data available to later make of it what they will?
ps whoopycat,
I checked out the stats table but it may as well have been written in another language. LeBron was the only recognisable name. |
Could not agree more, but isn’t that what Toole, Olive and co have been doing at Harman for the past 30 years or so? It seems I am defending a point I’ve never made, which is that no one does audio science. Of course big corps are using science to drive product development, but we in the consumer side should not assume that a handful of tests alone should determine the desirability of a product. The only reason to buy a DAC based on measurements alone is if you are building test equipment, or using a DAC to drive motors. If you plan on listening to a DAC at home this is foolish and lacks the scientific merit you think it does. Best, E |
Which one of these is the one that made me choose the speakers in my living room system?
"• On-axis frequency response
• Impulse response
• Cumulative spectral decay
• Polar response
• Step response
• Impedance
• Efficiency/Sensitivity
• Distortion
• Dynamics"
Or maybe it was some combination of those. If that's the case perhaps someone could share how they are weighted and what the metric is (was) for establishing that. |
in basketball, one thing I can tell you is that stats are very deceiving.
just listen to announcers give credit to players that do things that never show up on stat sheet. however, measuring human players performance is vastly different than measuring equipment that humans then react to the output.
So Erik, perhaps testing using a panel of 10 judges (made up of responders to your thread) listening to an unidentified speaker and using a 1 to 10 rating card. You end up with results like "speaker X got a rating of 89 out of 100 by our blind panel". What does that mean? maybe as much as the scientific measurements.
whoah! wait a minute..."blind Panel" - that's it! when you lose one of your senses the others become stronger or more acute?
regarding Bose, when I heard the 901's, they sounded muffled as if the speakers were in the room next door. however, when I got an Acura Legend with a 6 speaker Bose set up, it bested ANY sound system I heard home or car till that time. The Bose system in my Infinity sounded like crap. the Bose system in my MB sounded like crap ( typical one note bass). so my feeling about Bose is that they fit the need for such a large audience so well, why argue with their success. As for their science, they didn't achieve this success by accident.
|
larryrs,
"Which one of these is the one that made me choose the speakers in my living room system?
"• On-axis frequency response
• Impulse response
• Cumulative spectral decay
• Polar response
• Step response
• Impedance
• Efficiency/Sensitivity
• Distortion
• Dynamics"
Or maybe it was some combination of those. If that’s the case perhaps someone could share how they are weighted and what the metric is (was) for establishing that."
Only you might know which one made you choose your speakers.
I think they all matter, even if I’m not exactly sure what step response is without googling it first.
Different designers will weigh these factors differently, and that’s why speakers don’t sound the same, but why wouldn’t anyone want to buy a loudspeaker with exemplary measurements?
They’re not the final word by any means but they might be a great starting point.
You know I think it’s a bit like stats for cars. As far as I’m concerned most of them already have acceptable 0-60 times, but what I want to see is long term reliability and cabin noise data (also battery range for all-electrics).
These matter more to me than acceleration and top speed, but they all matter.
|
"jacksky wrote -
in basketball, one thing I can tell you is that stats are very deceiving.
just listen to announcers give credit to players that do things that never show up on stat sheet. however, measuring human players performance is vastly different than measuring equipment that humans then react to the output."
Not quite sure why those are vastly different. As you state, in both cases you have the readily measurable factor (e.g., points per game for a player/frequency response for a speaker) plus those those factors that aren’t readily measurable (if you are a sports fan, you will frequently hear a commentator or writer refer to the "intangibles" when describing a players value or contribution). You can easily count points per game but not the intangibles. I believe that at least some of the speaker qualities that make one speaker better than another for any individual are the equivalent of a player’s "intangibles". As CD318 said the readily measurable factors -" are not the final word, but they are a starting point." Exactly the point. |
I own a pass labs int 250,I love it. The rest of your article I don't know, except bose which is the biggest rip off in all of audio.
|
I am still surprised by how many judge science by whether it is used for their own preferences or not. You cannot judge science by whether it is used to further an end you particularly want to have happen or not. You judge it by whether the following occur: - investment in research
- discovery
- hypothesis testing.
Having goals that suit you does not make it science, or lack of science. |
