How to assess and quantify system improvement and changes?


In one of the currently popular threads, a number of members (all of whom I respect) have made significant claims regarding system improvement.

For example: A recent post mentions "a double or triple improvement in my system." A conversation a few days ago with two audio buddies had them quantifying a DAC change as a 10% improvement by one and 30% by another.

This, by virtue of our pursuit, occurs throughout our discussions in the various threads.

Starting this thread to see if we can mine the collective and come up with guidelines and outcomes that are reproducible, relevant and reliable for comparisons and discussions between audiophiles.

Thanks!
david_ten

Showing 2 responses by twoleftears

Good luck with this one.  Audiophiles almost by definition deal in hyperbole; to become a reviewer, first you need to pass a test in hyper-hyperbole.

Notice how systems are so often "transformed" by substituting, e.g., wooden speaker cradles for ceramic ones...

There's a fundamental problem here, as there's no benchmark that any of the judgments can be keyed to.  These are subjective judgments being made by individuals about unique combinations of components in unique listening spaces.  Try codifying that!

From a different perspective.  Replace a small, inexpensive 2-way with a large, high-quality floorstander.  More bass, right?, and better sound generally.  How much more bass?  How much does the increased bass enhance the overall listening experience?  (These are two different judgments.)

Then replace some throw-away 39c ICs with really expensive ICs.  Is there are much of a change?  Where?  In what areas do you hear it, and how do you quantify it?  Compare experiment 1 and 2.  Does the change in 2 equal that in 1?

Most improvements are incremental. People exaggerate, often without knowing they're doing it, to make a point.

A few years ago I dropped a pair of $200 speakers into a system that I'd previously been running with 5K speakers.  What struck me most was how decent, relatively speaking, they sounded.  No, not as good as the 5K, but as everyone always points out, not nearly 25x as good.

What I will say is that reviewers' hyperbole set people up for hearing more significant improvements than they actually achieve when they bring the component home.  Then the search for mitigating circumstances begins...