How to assess and quantify system improvement and changes?


In one of the currently popular threads, a number of members (all of whom I respect) have made significant claims regarding system improvement.

For example: A recent post mentions "a double or triple improvement in my system." A conversation a few days ago with two audio buddies had them quantifying a DAC change as a 10% improvement by one and 30% by another.

This, by virtue of our pursuit, occurs throughout our discussions in the various threads.

Starting this thread to see if we can mine the collective and come up with guidelines and outcomes that are reproducible, relevant and reliable for comparisons and discussions between audiophiles.

Thanks!
david_ten
@inna  funny and thought provoking.  And I get the point. : )

@ebm  I'm honoured. I figured a "Let me think really hard about that and get back to you tomorrow!" : )

Seriously though, take a stab at it if you feel it's worthwhile.
I’ve been guilty of hyperbole also, but I realize I was doing such a disservice to so many. Now I try very hard to put things in perspective and frequently say, "there are no absolutes in audio, only preferences. In addition, I submit at a certain level night and day differences don't exist.
I would love it if someone could make any headway in this endeavor, but I know of no way to quantify subjective measures of performance. This "equation", if you will, would probably have to be broken down into many parts with exact definitions of all the terms involved. I wouldn't even know how to begin.
I think this is almost impossible to achieve, because it is always based on subjective assessments.  We don't all weight various attributes the same, so, if I were to make a change that to me seemed to be a 24.8% improvement, someone else would say, no, I clearly hear a 32.3% improvement.  For me I am satisfied to place improvements in the following categories.  1)  Immediately obvious, even with casual listening. 2) A clear improvement in one or more attributes based on several minutes of critical listening.  3)  Relatively minor improvement revealed by careful listening to music that I have used for years to assess system/component performance.  4)  I just can't hear any difference, no matter how I try.  5).  Deoptimization in one or more attributes. 

good luck with this one.


really! I wish you do figure it out, but the variables in this quest are innumerable.


the predominate issue is Audiophillia Nervousa is a self diagnosed disease. 


as such, EVERYTHING then becomes purely the beholder’s viewpoint or assessment on every account.


in fact, the. variable factors extend farther than the equipment itself. one can have the exact same gear in one setting, remove it from that venue, transport it to some other home, or setting, in some other likely dissimilar room, and the results will probably be perceived differently. further, merely purchasing either a single piece, or even a ’turn key’ setup you have auditioned at a dealership, and put it into your own arrangement , and chances are it will sound different.


another separating issue is simply the fact few if any outfits will have identical pieces in tow. how then, can a proper account of how ABC aids or detracts from, any particular rig? or for that matter how good is the ABC gizmo on its own merits  ever to be reviewed?


if the aforementioned items are not enough to frustrate this affair, add in our ears are not hearing things exactly the same way or as well, as other ears!


OK, then bring in the meters and micrometers. of course! but which meters? which rulers? which scales? how will this or that actually get measured?


then too, which words will. be used in surveying or appraising the thing? humans are barely able to communicate these days and suddenly we are all supposed to be crystal clear oh the piece, context, syntax, and findings?

 

synergy. is yet another factor which will ever detract from acquiring a status quo for quantifying performance in terms of gains and losses. its a proven fact, not everything ever made plays well with every other thing ever made.


I’m not even mentioning circumstances surrounding electrical power anomalies which vary  from one installation to another.  . that is another bag of worms all by itself.


in years past, telephones were in short supply. regularly, to talk to a friend or relative, you very often  had to  travel to their home. In so doing one experienced at least in one fashion, the ‘charm of distance’. 


back then distance was often a desired boon for its yield was greater privacy. 


privacy used to be highly valued, now everyone wants everyone else to know what they ate for breakfast, or where their cat  coughed up a fur ball. and when.  


in the often tail chasing hobby of higher end audio, the ‘charm’ lies in the preferences and perceptions of the one paying the bills to be the boss.  his or her insights on individual items, or as an ensemble are the only views that matter.


the only rule of thumb I’ve discovered which generally points to  performance, though not empirically, is simply the ‘cost of likewise  goods’.


gear makers on average have been quite studious on pricing their products, mostly.  subtracting those whose view points lend to them over valuing their equipment, tier pricing seems to equate to tier performance. IMHO


paying to get to some of these tiers of upper end  performance can  bring on many tears. gnashing of teeth and wringing of hands are other participants in the aftermath.  especially if system synergy is not elevated from the alteration or addition of the newly arriving  new bright and shiny gizmo


exactly where these supposed ‘tiers’ separations exist is yet one more facet that lends itself to the charm’ or chagrin, of this hobby.


bear in mind, the notation on the ‘self diagnosed’ theme. 


charming, isn’t it?