I am curious how others have made A/B comparisons within their systems. What errors are encountered in this test? How do you avoid them? I often think of my stereo system as a pair of ski goggles. Have you ever worn a pair of amber ski goggles all day and then been shocked at the colors presented to you when you take them off? How does this phenomenon translate into the realm of sound?
Same mechanism. Neural adaptation and it is a common source of bias in long term even more than with A/B comparisons.
I recall a study done at a hifi show. Subjects made a blind preference choice between two options (speaker vs. speaker, or tube amp vs. SS) in a level-balanced setup. In every two-way test, the subjects preferred the second choice, regardless of what it was.
I find that A/B comparisons require time, especially when performed when in one's own system.
Sometimes, just the fact that something sounds different can lead to one feeling that it either sounds better or worse, initially.
It is only over time that we can determine that the difference is really better or worse, provides more of what we want or less of what we don't want (or vice versa). Many people A/B in stores in short listening sessions. To me, the problem with this can be multi-fold, but typically evolves around an evaluation that is based on too narrow of a field of characteristics.
Also, I find that if one feels they have a deficiency in their system, they tend to over-correct for that deficiency and sometimes at the cost of other areas of importance.
So A/Bing is necessary, but is part of a longer term process. The good news is that Agon allows for longer term A/B testing by smartly buying used with the ability to sell at zero to little loss.
FYI - I am in this process currently. Having recently brought in an Esoteric and Levinson integrated CD players (with Dacs). I will also be bringing in an additional amp and preamp. Giving me at least two or more of each component and hopefully allowing me to find the best synergy in a final system. I know this will take some time, but I also know that I can do this and then sell what I don't select at a reasonable end cost to myself.
Not being able to A/B components at home ends up being more costly and keeping one on the merry go round for too long.
It's almost impossible to AB anything without bias, for a variety of reasons... 1. When presented with 2 options, the second is almost always preferred because we naturally tend to evaluate the second relative to the first (but not the other way around). Since the second is usually the 'new' one, we are biased to the new. 2. Warm up time. As most know it takes 30 minutes or so for something to warm up. The interim delay is too long to retain realistic recall. 3. Visuals really matter. We are strongly biased to something that 'looks better' whether we realize it or not. 4. We are biased to the new and/or more expensive, again whether we realize it or not. 5. It's impossible to get a good sense of things in a short time. Long listening sessions are needed. Again, by the time you've listened to the second for a while, you've completely forgotten what the first sounded like.
Not sure it's possible to truly compare in AB testing. Of course huge differences in sound are readily apparent, but subtle ones much less so. Lesson - don't make changes unless they are huge ones. Small 'tweaks' are purely emotional.
What I find interesting is that fact that some changes that I've made (DIY speaker cables for example) seem to have resulted in a positive result the majority of the time, but a few song really don't sound right anymore. How does one choose what music to use for the comparison because each recording had different characteristics. When I added the Nordost Sort Kones I only noticed a positive result across the board, but maybe that placebo talking.
Placebo effect. My brother went to audio store to buy Cabasse speakers. Salesmen presented first speaker that sounded very good. Then he played a little more expensive speaker that sounded much better for a little more and finally even more expensive speaker that sounded incredible but my brother could not afford it. He then left my brother in silence for a while taking care of another customer and came back with another option - brand new Cabasse design that plays almost as good as the latest most expensive one, but cost much less. After listening - my brother said that this is a great bargain, sounding almost as good as the most expensive one, an bought it. At the end salesman told him that it was the same, least expensive speaker he listen to first.
"i just change stuff. period, and don't worry about it"
- So do I. I found that switching A/B is not as good as playing one component for a week with different types of music then do the same with the other. Constant switching between components to see what improved and what not has less value to me than general impression after week of listening.
Esthetics arenÂ’t always something of primary import to folks. Especially me. To a degree sure. They seldom play a part in my decisions as to cabling, or isolation.
Take cables for example. No longer do I bring in several brands or models within a brand or brands, at any one time, to examine, audition, or determine which one IÂ’d want to buy next.
I bring in no more than two. Usually of similar or predisposed application or intended purpose. Often IÂ’ve never seen either of these cables. I canÂ’t read the print on them and always ask for the maker to place tape on their downstream ends, well save for power cords. Lol
So I’m going “in the dark” or flying blind during my tests usually with cables.
I take notes. Use the same software, listen at the same times of day, etc., keeping my regimen as close to what IÂ’d normally do.
I try one sort or set at a time. For days and days, perhaps a week or well past the run in times. I don’t swap ‘em in and out readily. Although I can actually A/B two pr by going out of my DAC which has two active outputs, and switching from one source selection to another on my line stage preamp.
As I have more than one ready to go system, both sets of cables are reasonably up and ready most often. The interim for critical listening therefore is minimal.
What I feel is missing from the seemingly more immediate, A/B switching I could do, is thisÂ… warming up time. Sure I can swap from one input to another wherein only the cables are the difference, not the source, amp, speakers, etc. But you see, I am not including the greater length of playing time the one set has over the other set of ICs. So I keep to the above mmethodology. Try one item at a time and take notes.
The things that confuse me, or dilute these auditions results are simpleÂ… IÂ’m testing more than one item at a timeÂ…. Not waiting for the item to be fully run inÂ… not using the same musicÂ… not taking notesÂ… not putting the orig item which is possibly going to be replace, back into the system and listening critically once more as if it were newÂ…. Being in a hurry due to the setting, circumstances, or an all too brief period to make a decision was allotted by the dealership, or the maker of the product.
If IÂ’m not comfortable with my decision I wonÂ’t pull the trigger.
Normally, if no peeking was bing done prior to the first listen, youÂ’ll notice the obvious diffs right away. The real trick is to figure out if the diffs are better or worse diffs. All that matters is you like themÂ…. Or not.
Another item of concern should be going out to live musical events, periodically. Outside or inside. But ‘live’ for sure! It’s good to flex the eardrums, not to injure them, by getting out and hearing new music, and environments. It’s about as important IMHO, to buy music you don’t ordinarily buy. It also doesn’t hurt to run a test disc now and again, and take some measurements, especially if you run an all tubes setup. Or to check the bias if it’s checkable.
I bought one pr of speakers a goodly while back and wonÂ’t mention their name here, which I heard and loved at first audition. Absolutely loved that sound! Hands down, the best sound IÂ’d heard in my life. It wasnÂ’t the speakers so much as it was the front end making the audio signal.
These speakers were as well half the price or a bit better thn half the price of some other's I had just spent listening to for a couple hours. Same brand, different models.
Later I bought those speakers recalling their sound and thinking to myself, “those units are capable of outstanding sound! All I gotta do is give them good source and power.”
I made a mistake then. I tried to emulate that $30K system's gear, front to back, and was unsuccessful. Completely. It took me about two full years of avidly trying this and that to get them to work well. Once I DIDÂ… I SOLD THEM.
Live and learn. I know now, whatÂ’s in front of the speakers is as or more important than the speakers themselves.
One mistake I actually avoided was this, Â… I went to a particular local dealer and waited for better than an hour, nearly two, and then I was given a five minute demo of music THEY selected, while attempting to buy a $3500 Ayre preamp, new. In fact the salesperson would rather not have turned the thing on I believe. Half a dozen times he asked if I would buy it that day?
My response was always the same, “That’s sort of like buying shoes without trying them on, isn’t it? If I am unable to hear it I’m sure I won’t buy it. But if you turn it on and let me hear it, you stand a far better chance of getting a sale than if you do not.”
They did not allow for in home auditions or free trials, but if brought back in less than 10 days, theyÂ’d only charge you I think, a 10% restocking feeÂ… and credit the balance for some other purchase.
For whatever reason the salesperson didnÂ’t seem to care for my answer. Although I had the cash in my pocket, and it was a very popular preamp at the time, I didnÂ’t buy it. Nor will I go back to that dealer. Ever.
Following that incident, I stumbled across the opportunity to buy my current line stage preamp, and feel itÂ’s the best purchase sound quality wise, IÂ’ve ever been acquired. So in essence, it all worked out for me ultimately.
Blindjim - wow! Thanks for the answer, everybody. It's a strange fish to fry, isn't it? What I do is forget all about A/B'ng. I'm more with Elizabeth on this one. I just make a change. I DO NOT LIKE Stereo Stores and if they're scared to turn it on, what about me? Let IT GO! So far I have been moving towards the dry which is nice in a wet climate? Seriously, though, thought does go into it. I read in another post a serious exposition that to know less is more - or ignorance is bliss?? What I know of musicians minds is that they are better at integrating the R and L halves of their brains than just about anybody else. (Could this be tested with the Stroop test?) I'm confused by things that don't make sense, and there are a lot of things within music that do make a lot of sense. They are usually what the music is about -impressionism aside. I'm sorry to digress. These are source issues. So what do YOU say?
Sure, this is a broad topic... It's a struggle for me to remember what it was like before I was married. But what I am interested in here is comparisons made within a Stereo System. I do think that my impressions at least 1 week after a change has been made are more useful than my initial impressions - at least sometimes. This is one of the areas where it gets sticky. There are so many variables that initially a change for the worse may be a change for the better. But where do you get off this wheel? For example, if you add a cable from the D/A that dries things out - it may sound worse, but is that a good thing? The most I can say is that the more I am aware of the effects each component has on the sound the more I can perhaps predict the change the swap will make. In the end is there no reality to this artificial medium? I hear that 78's are awesome! I have some experience with Mono and I have heard at times more reality with my system in that format than I have ever heard from stereo.
However, IÂ’d suppose, wherever you feel suits you. Where You have no further desires to implement change (s). Be it at that point a blissfully rendering musical system is what you then own, or youÂ’ve just plain run out of enthusiasm for itÂ’s hunt and acquisitionÂ…. Or again, youÂ’re finally OK with what you have in front of you.
I think being happy with what we have instead of wanting what we donÂ’t have is the deal. How obsessive compulsive you are plays into this as wellÂ… as does how deep are your pockets and your patience and tolerance levels.
As for the system matching aspects, I would bet very very often, If I took a few pieces of my system to some other personÂ’s house, and inserted them/it into his or her rig, the results could go either wayÂ… better, so so, or even worse than was the norm previously.!
I try to optimize my systems as per their individual needs at the time, based upon my perception of itÂ’s shortcomings, and my own preffs for hearing music. Consequently, introducing some parts of my rig to another can alter the secondary rig to sound brighter or dryer, as an example, perhaps. Such results arenÂ’t always the case of course, but could be.
I think itÂ’s about optimization. The same piece of gear setup one way on one system, can produce different outcomes on some other arrangement and setup. I also get the overwhelming impression that many hobbyists strive towards a certain sound. Not necessarily a flat and neutral sound throughout. I did. I donÂ’t now though. I went from one extreme to another. From crystalline, bright, analytical and vivid, to warm, dark and liquid. Now to as much resolution and detail within the confines of the music being played that it remain quite revealing, but musical as well. So I now push the boundaries of the gear I possess. Eeking out as much info as it can deliver, while adhereing to musicality as the deciding or determining factor if push comes to shove. Always IÂ’ll choose musical ease over uber revelatory reproductions. Riding the line in between them seems more my concern latelyÂ…. And IÂ’m very satisfied with the results I currently enjoy. Only a couple finer points remain that IÂ’d care to address, or add to the fray. Like a dedicated 7.1 processor, and another power cord or two perhaps. Maybe some replacement outlets as well, by and by.
Consequently, IÂ’m in no hurry to apply these ideas or add those desired items. Therefore, I feel IÂ’m a pretty happy camper lately with regard to the results I get regularly from my main stereo and mostly from those others I own..
Because itÂ’s a past time, a hobby, some will continue to try a this or a that as they come out. Just for fun maybe, or for personal enlightenment. Possibly for bettering their own gig. ItÂ’s just another facet to this sideline. Whichever way one goes, with earnest sincerity, or as cavalier as one could imagine, itÂ’s a hobby. Not a race or competition. It should always amuse, entertain, interest, possibly fascinate at times, and ultimately satisfy the enthusiast along the way or at least eventually. If it does not IÂ’d submit they find some other activity that will.
Everything matters. How much everything matters and in what way, is the curiosity that drives this and many other hobbies.
Some years back I bought a $300-ish NAD cd player for my 2nd system. Just out of curiosity, I did an a/b against my twice-as-expensive Music Hall which was my "good" player at the time.
Based on that comparison, I thought the NAD did most things better, and after switching back and forth for a couple hours, I decided to keep the NAD in the better system, and put the Music Hall upstairs. A couple weeks later, I realized that I was never listening to entire cd's anymore. There was something missing with the NAD in place. It sounded good, I always got bored quickly.
Eventually, I switched back and found that I enjoyed the system more with the Music Hall, even though I couldn't identify anything it did better. So whatever magic that player had was apparently too subtle to pick up on when doing a/b comparisons. Lesson learned is that for me at least I've got to give a component enough time that I'm really just listening to the music.
By the way, I still have both players, and it's really a crap shoot as to which will work better in a particular setup. Turns out the NAD isn't inherently boring-it just was in that particular setup.
Last night I went to my son's Violin Master's Class. Small venue - maybe 10 to 15 people present. Piano and violin, that's it. I have been listening to my digital source lately at home. I'm having trouble figuring it out. I know it doesn't sound real to me, but why? What I am getting at is last night I feel like my ears woke up - again. The real sound is hard to describe but I feel that I am developing a sonic memory for it through training. Ear training. My A/B comparison is moving toward Live vs. Recorded. Live is not a shifting reference. It is repeatable. The Master's teacher was speaking about chords on the violin last night. A single note is in tune with itself. Only when compared or contrasted with another note can it be said to be out of tune. To play a chord correctly on the violin, hold one note fixed and adjust the other to tune the interval. So I agree. Does your system sound like live music? Time will tell.
The recording process is the first step away from live, and the best equipment can keep that damage to a minimum down the chain, but once recorded it cannot be resurrected IMHO. Sure my system sounds like live music (unamplified acoustic), but I never confuse it for the real thing - A/B or Z.
How about when you shut your eyes, can you imagine that you are in the venue? Sometimes I can. With digital it's like I'm there but pixillated. With analog, sometimes it works but usually there is smudge.
My latest CDP modification includes an A/B switch that enables on-the-fly comparison of RBCD processed through a 13-year old DAC chip to RBCD processed through the latest 32-bit ESS9018 Sabre chip. The difference is meaningful but much subtler than any audiophile would expect. That A/B switch cuts through a lot of cognitive dissonance.
A/B tests are a useful tool for determining differences in sound when changing a component, but one should not rely solely on them.
You have to listen to a single configuration for an extended period of time, get the setup optimized again after each change and listen to a variety of recordings of interest before you can really get the full picture of what a rig sounds like. Then go from there, including a/b tests along the way as needed.
If a product is superior, ANY person will readily hear it. All the excuses of why A-B testing can't work, is a "cop-out" when you drive 2 cars you can tell which is faster, and handles better.
Especially, on a "track".
Audio, is "Subjective", but bring the ladies in and the b.s. all comes to a stop.
For the most part. Besides, it is a well known fact that most women hear better than men do.
Ask your Hearing Doctor.
You might get your feelings hurt, but they will tell it like it is.
A $10k amplifier, should be better than a $500 dollar one, if not, it is a "rip-off".
A quad core computer, will run more programs faster, than a single core computer.
Audio, is the single biggest "hype" IMHO in this world.
When people come over my house, that are not "Audio" lovers at ALL, they without ANY hesitation, say, "they have NEVER heard music sound so real", women, the U.P.S., Fed-Ex, drivers,just regular folks, etc...
When they ask if they can come in, and sit and listen for a minute, and 45 minutes later, they are saying, "I gotta get going, or I'm going to get fired..." because they are in "Awe", listening to music like they say they "have NEVER heard Music sound like before".
So, for me, I know that I have finally "arrived", because I don't have to pump people, saying "do you hear it?", "doesn't it sound good?", as soon as anyone comes to my door, and they hear the music, from outside, they say, "how real, and dynamic the music sounds".
But, so many times, in the past I have been invited to a "Audiophiles" listening room, and all they do, is "pump" me, over, and over, saying..."have you ever heard anything sound so good?"
Like they are trying to convince themselves?
When my wife is with me, she mixes no words, no matter how fancy the set-up is, ALL she cares about is the SOUND, PERIOD!
She has brought so many grown men, to the brink of tears, due to her brutally honest comments.
That's why most of the time, the guy's house I am visiting, his wife is always out "shopping".
She won't blow "smoke", if the sound is crap, she will say so.
Needless to say, rarely am I invited back. (I am crushed) NOT!
So many people are in to the "Looks", and not the "sound", we hear with our ears, not with our eyes.
But, so many people decide how stuff sounds, even before they hear a single note.
If they see "Goldmund" right away, no matter what else, is with it, they figure it's going to be "Gold", but that is not always true.
It is the "sum of the parts", not 1 piece of equipment "makes great music", rather the "Synergy" of the components.
Probably why Audio stores are no longer around, and most Audio, is sold out of someones home instead.
For years Audio has gotten by because of 2 Magazines, what they wrote, either "makes, or breaks" the product.
People used to just buy stuff, because one of these magazines gave it a "rave review", often the person would go into the store, without even listening to it, they would buy it because of the review.
So many times, I have witnessed purchases made this way.
Later, it is for sale here on Audiogon.
But today, with gas so expensive, most people, are not as easily swayed, as they used to be.
When you go listen to a $100k system, you are not supposed to come home, and say, "wow! my setup sounds better!"
Or your wife tells you, the same thing.
When people that are not into Audio, take real notice, at what you have, the sound I mean, then, you know you have something special.
When they say, "my stereo sounds terrible compared to this!"
Everyone, has sold something that they wish they held on to, after buying something else to replace it.
I'm with Bjp, sort of: If you have to squint to see it (or rather, hear it), don't pay for it. Because of the psychological factors people have mentioned, to say nothing of the difficulty of doing A/Bs where everything else is constant, I have near zero confidence in my judgment when the differences are less than pretty dramatic. I also find it useful to do comparisons with a non-audiophile, be they male or female, since they may be less motivated to detect "small but significant" differences that may not really be there.
No doubt, others are more discerning, and they may reliably make small magnitude discriminations -- and be willing to pay for them!
Agree with a few of the above posters: Snap judgements are frequently wrong. Time is, in this case, your friend.
Differences may be easy to hear, but better?
Just from a setup standpoint, I make many changes fairly quickly when doing a fresh install. Than I listen. Tweak/Listen for a couple weeks. Let it sit and listen some more. The last few changes? maybe a couple weeks between.
Oddly I find my A/B work is only really let me down when I really wanted one option to be better. Usually it's a factor that appeals to my engineering brain, aesthetic brain, or cheap skate inner self. When I am honest with myself I have been satisfied with my comparo's. I have been pretty good on the audio side but I'm completely useless on the guitar/motorcycle side of the equation. Last couple audio A/B's I did indicated that I could hear a difference but neither were enough to warrant the cost in my opinion. Went home happy with my existing system.
You must have a verified phone number and physical address in order to post in the Audiogon Forums. Please return to Audiogon.com and complete this step. If you have any questions please contact Support.
