I'm not sure where this notion that audio precision is sceptical of their products ability to measure. Are they looking to improve their products? Certainly what company isn't. But if this idea has come about from a YouTube circulating from an engineer from AP then you need to watch it again. He isn't saying their device is lacking in measuring but that some companies cherry pick measurements performed by an AP device so consumers need to be wary when looking at company advertised SPECs which are not the measurements.
Amir and Blind Testing
Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement. Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer. We are too affected by our biases to come up with any valid data. Maybe.
355 responses Add your response
| Post removed |
What I take as Amir's main point seems to be lost among the personal attacks on him, most of which appear sadly mis-informed. What I hear him saying to the manufacturers of equipment which does poorly in his tests is "show us your data." Or conversely, "admit that you have no data." That's it. Simple. Then consumers can decide. What we see from some of the worst offenders is a lot of technical sounding talk which suggests a technical underpinning to their claims but zero data. Did they design their product with no measurements? Perhaps. Then tell us that. I picture a cable maker for instance that soaks his wires in vinegar then declares they sound better to him. Well more power to you buddy! But if you want me to buy it you don't have to divulge your proprietary secret, just give me something more than techno mumble jumble. If indeed the improvement cannot be measured with current technology (a possibility I am willing to concede) and the manufacturer is not willing to foot the expense of well conducted, objectively fair listening tests to support their claim then I guess it is a true "audiophile grade" product. Does it makes things sound better? Depends on who is doing the listening and what they ate for breakfast and whether they tripped over the dog while leaving the house this morning, I guess. |
I think he is measuring the wrong thing. Ask your guru to show how he measures a person's emotions. Some folks prefer precision while some prefer musicality. Now you would argue that you prefer precision over musicality - and I get that. But what if the "happiness" I get from musicality exceeds your "happiness" about precision? How would he measure that? Maybe he should try to listen without prejudice/bias. God has given him his ears - why does he not trust them? |
Precision is mainly a concept for electronical engineering... In psycho-acoustic precision is subordinated to musical experience and human ears evaluation... And the precision did not exist for itself or for delivering an electronical product but to understand subjective hearing experience... Amirm is not even wrong about audio ...It is useful to have the RIGHTFUL measured data for sure... Myself i trust the only fundamental science for my audio system experience: it is not comparing amplifiers with some measuring tools...IT IS NOT ENOUGH AT ALL ... I dont say it is not useful. it is... But acoustic and psycho-acoustic are the main fields if someone want to understand hearing and music experience and improve at low costs his sound system/room ... Amirm sell digital technology of all kind not music or acoustic or the optimal audio experience...He is a digital tool specialist...It is what he sells... Like i said he is honest, i dont think he is not at all, but his disciples in audio are deluded and obsessive, they dont have the excuse to be digital engineer like their master... ...
|
"Ask your guru to show how he measures a person's emotions." That is a little pejorative, don't you think? Your other points are fine, but your religious references make me think you might be investing this discussion with more significance than it deserves. And no, Amir is not my savior and he is not yours either, he is just a guy testing audio equipment and sharing info and asking others to do the same. |
@bruce19, Not at all. I asked a simple question. Why did you think that was pejorative? What religious reference are you talking about? I have absolutely no idea how you came to this conclusion! Please explain, if possible, since this one really puzzled me.
You see, 99% of the folks buy audio equipment to get the joy of listening to music. I have been to many a audio shops, multiple shows, etc. All of them demoed me how music sounded on their equipment. and I loved some, and some I did not care. Of course, I used my ears and that is how I chose my system that you currently see in my profile. Not one demo consisted of measuring anything and then making a decision to buy that equipment. Don't you think it is a bit strange that someone would buy a equipment just to "test" it and not listen to it? |
It is the reason why i called them : measuring tools fetichist... There is also the brand name tasting fetichist group... The two groups underestimate and dont give a damn about acoustic...the most powerful known upgrade is acoustic and psycho-acoustic devices...
|
Actually- this thread was not about electronic measurements. It was about Amir’s observation about the human brain and its ability to hear non-existent improvements when switching components because we start listening in an “analytical mode” when we are testing hifi equipment. Of course, Amir is not an expert in psycho acoustics, but I thought it was an interesting concept and many members related to it to some degree. The human sensory system is so easily fooled. Ever take one of those flight simulator rides at Disney? Vibrate the seat, tilt it back and show us a video of flying and we swear we are traveling through space. A little scary how easily our senses can be deceived. What do we do with this knowledge? Whatever you want, that’s what. |
SINAD is useful but it's not the only thing measured. It seems to me the amount of half truths and downright nonsense posted here about ASR borders on paranoia. Perhaps it's because they are a growing presence and some feel threatened but after all it's just another website the deals with measurements and objective and to some degree subject aspects of high fidelity playback systems. You dont have to take everything presented there as gospel but with an open mind you might surprise yourself as to what you might learn. |
Thanks for the reference... Very interesting set of articles ! But i was talking about small room acoustic and psycho-acoustic of small room... And it is a field in itself... And if you read the site all is about psycho-acoustic and digital technology.... Almost anything about small room acoustic... What matters for them, at least for Amirm are the tools to measure human hearing in sound PRODUCTION... Not about sound ACOUSTIC TRANSLATION in a small room...Signal processing is not room acoustic.... I dont suggest here that this prycho- acoustician who work with them know less than me... I am not an acoustician myself for sure... I only express my opinion about the emphasis put on DIGITAL TECH. and hearing abilities in the digital tech.context, not in the relation between physical acoustic and psycho-acoustic in small room , which is a field in itself... My point is that ANY measures of the gear must be subordinated to the small room/ speakers relation and specific ears abilities and listening history of the owner of the room... Because CHOOSING a piece of gear at his peak working potential ask for the guiding listener/customer subjective impression in a room not only about some electronical measures of the gear...A point that NO ACOUSTICIAN can contradict by the way.... Even this one....
In a word what these articles spoke about centered around digital tech. EQUALIZATION, my own approach to room acoustic centered around more Primitive so to speak, Helmholtz MECHANICAL approach... I will read these articles because psycho-acoustic dont change his principles from the digital field to room acoustic... But in digital tech. the focus is in the technology itself, in room acoustic the focus is around the owner of the small room... Some subjective impressions are at the center of the matter here; AFTER an objective disposition of the acoustic content of the room in relation with the specific abilities and limitations of the room owner...
|
In a word i dont need a blind test for any change in the acoustic disposition of my own room... And the specs sheets of any piece of gear so important and useful it can be to pairing piece of gear with one another SAY NOTHING about the FINAL sound quality of the system as a whole and his relation to the specific room and to my SPECIFIC ears... Simple no? I reject the " brand name gear tasting subjective fetichist " and "the digital tech measuring tools objective fetichist " TOGETHER here , because i live and listen in my room acoustic and the two groups completely UNDERESTIMATED small room acoustic power....It is there in my own room that the gear succeed or fail FOR MY EARS , nevermind his measured specs...Correlation between the objective position,size, tuning of acoustic devices in my physical room and my own tuning subjective abilities and preferences are WHAT MATTER.... I called what i added to the classical passive materials room treatment, mechanical equalization and active tuning control of the room after Helmholtz method... I dont design a product to be sell, i design my own room.... AT NO COST or almost... 😎 But i made it with a METHOD.... Why? Because a SMALL room is not an amphiteater acoustic....One is designed by my ears for my ears, the other one is designed for a crowd... Timing of waves, pressure zones control distributions for example dont play the same game here...
Acoustic of the room is the sleeping princess, all pieces of gear are the 7 working dwarves with their tools , and psycho-acoustic of my own specific ears is the kissing prince....
A riddle: Do you think i modified my speakers for example in reading only their specs sheets ? or by listening them in my room? I listened to them and used Helmholtz resonators and diffusers ON THEM also to crossfeed each one and decrease the acoustic crosstalk in my own way mechanically...Results are astonishing...In near listening and in regular listening... Each one of our ear and singular speaker is a world in itself....
|
I just rebuilt recently my loudspeakers Xover with top Cardas Litz wire,WBT connectors, and $1500 in Xover parts , no factory even at $30 k are as good , my point mfg take many shortcuts, Magico A5, Marten ,using mid grade Mundorf Parts when Mundorfs top capacitors are their Supreme line Not Evo , the A5 is the first to use their Excellent New Ultra $$ resistors which is a good start But on a $25 k speaker you deserve much more , in our group our Magico friend is upgrading his capacitirs now . My ears are not just getting used to the new parts there is is vast improvement in imaging,soundstaging and detail another guy has the exact Dynaudio 3.4LE speakers and now he is saving for the same mods it was a profound improvement , used$3k now better then many in some respects then the majority of $10-12 k speakers . buying a Quality used component gives you great value and ability to truly raise the Audio bar !!
|
Thanks to you djones , i discovered not only these articles in acoustic but an acoustic thread in the ASR site which is more than interesting... Then yes Amir, is interested by measuring gear, but the site is more than that... The problem is not Amirm but some audiophiles fanatic disciples indeed here and there... And yes there is no less acoustic posts in ASR perhaps than in Audiogon and perhaps better one... Then... Then thanks to your post information.... It seems to me that my own perspective anyway is right about audio: correlation between objective and subjective perspectives... And the gear matter yes, but acoustic/psycho-acoustic matter more...
|
At least presumably the function of good audio components is to reproduce as accurately as possible any given sound, especially music. So measurements are a way of estimating the efficacy of the equipment. The performance of any given unit must relate to these measurements for it to be considered accurate.
|
As a side note to my own thread, many people knee-jerk into thinking that Amir thinks high-priced components are BS and cheap Toppings are the shizzle. Not so. He raved about the $4000+ Dan Clark headphones and he uses a $20k Levinson amp with $20k Revel speakers in his own system. He also raves about the $10k Genelec monitors. |
@chayro Just to return to your original question in this thread, as long as you switch between components when evaluating gear and your mental state (analytical, relaxed, mellow, angry, happy, silly, or whatever) doesn't change dramatically as you switch gear, you should be able decide which gear sounds best to you. @rtorchia Your comments about "accurate" sound compared to "distinctive" sound in terms of distortion are not consistent with what I have learned over the course of 50 years of listening to hifi gear. I've lived through the Stereo Review mindset of the 1970s and 1980s, when designers sought ever-lower levels of THD (usually by increasing feedback in the circuits), despite the fact that many of these products sounded harsh and electronic. Many of the comments on the ASR site seem to embrace a view quite similar to that now debunked view from Stereo Review. If you want to learn how to design gear that measures and sounds great, pay attention to the approach of a great audio designer like Nelson Pass. I recently watched a YouTube interview by Steve Guttenberg, during which Nelson discusses the importance of listening to distortion profiles rather than just measuring THD levels when designing great amps. Some people like to claim that amps from Pass Labs add distortion (mainly 2nd and 3rd harmonics) to improve the sound, but Nelson says that they simply don't try to suppress the 2nd and 3rd harmonics as much as they suppress higher order distortions. So which design approach is more "accurate" and which is more unnatural, one that reduces the influence of higher order distortions or one that simply reduces THD? I would argue that whatever sounds more natural and more like real music is the best choice. And Nelson Pass stresses that they use an ongoing approach of measure-listen-measure-listen to design their amps. Any approach that prioritizes measuring over listening clearly does not have its priorities straight. |
I think this is now the 3rd time I've posted a link to this topic over the past 2 years concerning ASR so here it is again. All the best, |
Quite so (along with your other observations), and I took the liberty of changing your quote slightly so that it has a wider audience amongst audiophiles. |
Perhaps (I thought it was more like the early 80s, and as always I am happy to be corrected), but I suspect that technology and knowledge has advanced since then to accommodate such characteristics in general without the pain of fingernails on blackboards. edit - folk who have amps from the 70s may recall, for example, amps such as Marantz, Sansui and many others being capacitor coupled. I’d like to see those (refurbished, of course) put over the measuring bench. |
@noske - as I recall, the Japanese receivers were boasting incredible distortion specs in the 70s, but the high-end stuff started becoming more well known in the 80s. And by well known, I mean by a very small percentage of the population who were reading the audio critic and absolute sound. Remember, way back then, most everyone had a stereo setup in their living room flanking the TV, but the early high end stuff was relatively rare. |
@chayro OK, granted, I stand corrected. I didn’t know that. I wonder what kind of gear they used at discos? Incidentally, I do notice from some pics here that some folk still do have their setups flanking the TV, perhaps for home theatre reasons. |
I'm glad you did because I missed the others. A lot of ASR fans may consider it tough and cruel but honestly it's compulsory reading. |
I didn’t say, but same here, 2ch stereo HT. I’m comforted to know that it has no measurable effect. (On a 6 foot long table from a school chemistry classroom. Its tall and the correct depth - I don’t like this kneeling/squatting business to push buttons, do cables etc. Blah) @chayro I guess I was talking solid state amps. I don’t know if tube amps ever pretended to have minuscule distortion measurements that the Japanese gear you mention were achieving. I wonder if the very low distortion amps would stand up to scrutiny on Amir’s bench with all the extra parameters we now know are important to sound quality.. |
ASR is pretty used to empty responses like that one. It basically says "I don’t actually have any good, civil arguments or evidence in response to ASR’s reviews...but since I still don’t like their conclusions...here’s a disparaging meme so I can feel like I got one over on them."
Embarrassing enough once. But..3 times?
|
milpai,
That's missing the point. In the case of, say, the Nordost USB cable or PS Audio P12 review, Amirm's measurments indicated no change to the signal that would be audible at all. Measuring someone's "emotional response" tells you nothing about what's actually happening in reality, in terms of the gear. I don't give a damn if you have an "emotional response" to a Nordost USB cable because that's you (and likely your imagination). I want to know if it ACTUALLY does something for the signal so I know what I'm spending my money on.
Some approach high end audio like they do a religion. Not everyone wants to do that; many of us want actual knowledge as to how the equipment works so we are making informed decisions with our money.
|
The inherent flaw of any sort of blindfolded A/B testing is when there's unfamiliarity of the test subjects with the items being tested. For example - a blind taste test of Coke vs Pepsi is adversely affected if the test takers aren't cola drinkers. Without a recognizable frame of reference, "best" or "better" is merely a guessing game. |
Great post! I concur... But "Objective measuring tool fetichists" , not Amirm perhaps, but his less enlightened disciples, will claim that sound experience, contrary to any psycho-acoustic/ physical acoustic science fields experience, will come directly and is DECIDED by and from the measured gear specs , not Amirm who is intelligent enough to give only his personal measured numbers, and will SUGGEST that his measures numbers had this meaning or this other one ...But for his disciples this suggestion is a defintive dogma... No listening experiments can contradict it with any value of any kind...Only blind test will defeat SUBJECTIVE biases....And they need to defeat it... But we cannot optimally tune a SMALL room for ourself WITHOUT our learned subjective biases with only objective physical acoustic principles ... 😁😊 People are gullible, be it techno fad "alleged" scientist or those other type of " fetichist who taste their brand name gear" in itself for itself without any objective context to put them at test....The more important context is a room acoustic controlled or not ... Audio for me is investigation by listening experiments of the acoustic/psycho-acoustic dimension... It is not about a McIntosh or Schiit or Mephisto amplifier specs or price... For sure all piece of gear are different by their design but what we can do to put them at an audiophile level of optimal working ? This is the question... There is a good amplifier at any price tag....New, old or vintage anyway... Deception for me in audio is the ignorance of acoustic importance...Bad design will exist even after Amirm measured testing tool bench test , and sometimes good design will exist in spite of his critic with his measuring tool... And anyway what is good in some room may be bad in another room SOMETIMES... Reality dont emerge from a simplistic formula.... There exist truly bad design but no measures specs are necessary most of the times, a little listening will do...
|
Hello, not much discussion here about statistics... well, it's not an exciting subject really. I thought the point of blind testing was the sample size. One person is not enough, maybe you need more than 100. If more than, say, 60% of them opted for A you might be able to say that A is better than B, and if it's 50/50 then that's a pretty good indication that A and B are close to the same thing. I would venture that when only one person attempts AB you will not get conclusive results unless one of A or B is truly terrible. It's hard to get 100 audiophiles into the same room, and you would need good security to prevent fights breaking out.
|
You are right for sure... Blind test is most used in the pharma business among others industry...There it is easy to understand why it is very important tool and testing drugs methodology ... Statistical method and blind testing goes hand in hand here... But in audio if even singular non statistically significant blind test in some case yes can be useful, CLAIMING it must be used systematically on people to DECONSTRUCT their CONTINUOUS experience in meaningless bits is bordering ridiculous when "measuring tool fetichist" warring against "brand name testing subjective fetichist " called it "science"... Audio is a science where A CORRELATION between subjective experience being maintained must be accounted for with objective tools and methods yes, but not systematically deconstructed by ideological claims...Audio experience is more encompassing than audio industry itself , it is not primarily about gear and tools it is about acoustic/psycho-acoustic and sound/speech/music scientific relation... Acoustic/psycho-acoustic first and last are sciences in audio with electronic engineering between these two moments of A/P.A...Blind test is a secondary tool at best here, not a primary concern like some fetichist claim for their weak argument about audio which is : We listen and can really listen ONLY what is measured or explained by some set of measures...The rest is deceptive fraud or illusions... Subjective Perception of tonal timbre for example may be tested by blind test usefully but not negated in itself to exist OBJECTIVELY with the utmost value even if it is a subjective determination... Some people are better than other at this task of timbre tonal perception for sure....Subjective evaluation controlled and correlated with and by objective means is the heart of music/sound experience and science but the reverse is ALSO true, objective means and tools must be subordinated also to the subjective evaluation itself , it is a true TWO WAY correlation process ...Not an ideology... Hearing is not even understood completely by far today... It is very revelatory that the two warring fetichist groups battle each other AROUND the gear, focusing on the gear MARKET, and not around acoustic/psycho-acoustic experiments and experience in audio thread... And like A.I. promotion in some circle is used AGAINST human real INTELIGENCE , which is conceptual creation, in audio digital industry some want and claim to redefine human hearing experience WITHOUT the need for a human subject.... This is not science in the two cases but ideological groups inside true science...
Welcome to you here by the way....
|
Until an appreciation of stats demonstrates how often they may be used incorrectly to draw conclusions that have no basis. That’s just a general observation, so moving right along.... If in fact there is a difference and 40% say there is none, this is saying that those 40% of people have less than optimal hearing. Isn’t that conceding that it is a poor test audience, not to be relied upon? If we are to trust out ears (whatever that means, despite it being some kind of mantra), ought not 100% agree that there is a difference? And if prior to the test an unknown portion of the audience cannot trust their ears, on what basis can it then be said after the test that A and B are in fact different? These hard of hearing people may be saying there is a difference when none exists. The good thing about the one person test is that the variability in peoples hearing is removed - if you have "poor" hearing, any AB test will be subject to a query and be discounted. If "good" hearing, then multiple tests would need to be done in accordance with good practice (not so easy, as it happens) - to find some measure of confidence would involve discussing probability theory but there does exist common sense rules of thumb which I don’t really like much. Clear as mud? edit - by the way, there is no requirement that the individual/s tested be into music/audio gear, whatever. The best test subjects would be teenagers or even slightly younger. Just sayin' |
| Post removed |
Under strict test conditions. This is a given. I do say that this isn’t easy. Common sense suggests that something may be learnt from a test when there is a difference and 40% says there is none. It is not a "dead" number - to an analyst it speaks information. What would be more interesting (to some, anyway) is where there is no difference and 40% (or 60% or even just 5%) said there is a difference. Could any conclusions be drawn from this? Perhaps the test wasn’t blind or conducted properly (this includes a person who is partial to the outcome conducting the test)? Hmmm. Correctly, this aspect is conceded in the comment. Given that ears are apparently to be trusted. As is often advocated by many good folk here. In any event, individual tests are preferred for reasons. |
Posting something like this which is the opposite of what’s in the link, imbues you with what, pray tell? A silly sense of accomplishment? All the best, |
This has gone assbackwards the device under test is the listener not a DAC, amp, whatever. The measurements have shown no difference in two devices say DACs, person makes claim to hear differences, the ABX is for the person not the DAC. If the person can pick one DAC from the other better than chance, which is where statistics enter, then his claim is taken seriously and viewed as objective not subjective nonsense about blacker blacks and wider soundstages. If a company wants to show their fuse, cable, smear on goop, DAC etc.... affects the sound then the type of blind tests being discussed using sufficient people shown to have excellent hearing etc.. would be conducted by a third party to avoid company influences. Not that any of these companies that claim this stuff will ever do them. |
A sound proposed in a blind test, and disconnected from the listener room historical intimate soundfield and proposed to be evaluated in a testing unknown room alien soundfield with a different relation with the speakers/system used cannot prove anything certain about the alleged difference claim by the listener... Why? Because the interpretation of a sound difference is ALWAY different coming from system differences in the test, or coming from the speakers/room specific differences for example... The way a dac translate a specific soundfield from a recording trade-off set and propose it through a specific system FOR another room soundfield is completely specific and always vary...The difference between sound coming from gear in different room conditions can even be huge... My dac cannot be judged to stay the same after changing the room compared to before for example and this is also true of my amplifier, cables, speakers, they give something which is the same by virtue of their designed qualities but in a complete different way when i modify the A./P.A. conditions..... I myself could not be able to detect all differences i can detect in my room with my gear transported in other acoustic conditions ... I know that WITHOUT blind test and with a CONTINUOUS Listening experience where the affective and acoustic cues are never disconnected and where my memory then can goes on working...it is how i tuned my room....ANY CHANGE CAN BE DETECTED, even sometimes a single straw acting like an Helmhotlz diffuser or a single straw variable perimeter and lenght working as the neck of an Helmholtz resonators.... But mark my word here, I have ORDINARY hearing ability in my age group by the way but an history of listening experiments for years in MY ROOM which explain how and why i can did what i did... Then mocking me with the false claim that i pretend to have bat ears is BESIDE MY POINT.... Proposing to test me by blind test make no sense at all....We learn to hear in some condition and with particular tools or instruments and devices...This is this content we learned to identify not by superpower but by learned habits in a KNOWN context with familiar sound/speech/files/albums... Blind test prove nothing generally in an audiophile tool fetichist deconstructing context... But those using these blind test to debunk some product are , like the opposite group those who promote these same products in the same gear fetichist war FORGETTING acoustic/ psycho-acoustic (A./P.A.) conditions and science... But when this is said, i dont think that Amirm measures are useless, not at all, but used by his disciples cohort of fetichist tool measured specs, which are not an absolute anyway, they go and stay in an audio dead end, ... In the same way audiophile focusing on the brand name gear goes in a dead end forgetting the relation between A./A.P. to idolize the gear piece overestimating his power contribution to the final S.Q. experience and undersestimating completely the crucial relation of the gear with A/P.A. conditions ... Prove me wrong... It is my personal experience i dont pretend to be a scientist, but i learn something in the last years in my room... 😁😊 The difference between piece of well designed gear in the same bracket of the S.Q./price ratio EXIST but are way less diffrence than the improvement make on them by putting them in the right acoustical embeddings settings... The diffrence here is HUGE....This is true for all piece of gear at ny porice tag... No speakers beat the room like said some unknown acoustician... This is true for a dac, an amplifier and for any piece of connected gear in the system...
|
prof
Is this just like how you want to imagine that every person in the world has the same hearing? Might the following article might help you? Scientific Study - not imagination Search the net and you will find many more scientific articles - not "religious" articles, as you mentioned. Then there is the wax build up, hearing loss, genetics, etc.
Fortunately I do not fall in that group. I use my ears for listening. And the brain translates that into emotions. Eyes are great to see the numbers and study the equipment. But what we hear dictates our preference and that translates to emotions we experience. If a scientific study cannot convince you that we all hear differently then only Amir can help you. Good luck. |
