AB Testing Experiences

Lots of marriages being tested herein.  Gives me chills.  

Yes you need to level match. Get a SPL meter and do the best you can with it getting volumes close. I've  done it with DACs and could never pick the right one though I do admit the Node2i has a pretty bad implementation. 

No Millercarbon I have not conflated anything. I said tonal balance. How we perceive a sound is based on how we perceive a whole set of frequencies that make up said tone or sound. That is absolutely impacted by volume. That is irrefutable. It’s how our ears /hearing system works.


With the same sound, louder volume does not mask quieter passages unless you are reaching saturation of the hearing system EXCEPT as described above the impact of tonal balance due to Fletcher Munson. Increasing the volume makes the bass increase in relative volume more than other frequencies which can have a masking effect in addition to changing tonal balance. You proved yourself wrong right there. You are trying to ignore biological reality to suit a particular position.

Listening experience can make you better able to extract information, but it does not change the basic biology you were born with hence why it is so important with AB to level match.

@mahler123,

It wasn’t MP3 v SACD, just CD v 320kbps.
Maybe I should have tried classical.

We did try a few high-res discs (Dylan’s 2001 Love and Theft was one) but the results were not convincing. It sounded a little soft (analogue?) and we weren’t too sure about the legitimacy of the mastering, or even which layer the Pioneer deck that we used was actually playing.


’I enlisted my wife to change inputs at one point, and she was nice enough to cooperate, but she was clearly bored, and it also led to the question on her part “Since they all sound the same to me,why do you need all of these different boxes?” so heed my experience and don’t go there’


Yes sadly, enlisting help is very difficult, not to mention opening yourself up to further ridicule.

I used to put the comparison discs label up and try to load the CD player with my eyes shut!

I know, this is certifiable stuff.
Luckily there were no witnesses.

A better method was to record them onto Minidisc and set the machine to random.

I still miss Minidisc, it was a fabulous format. Especially for all us Nick Hornby types who love compiling top 10 lists and rearranging them at will.

Once again I suspect those reviewers who kept kicking it without actually comparing it unsighted to CD had hidden agendas to do so.

Oh well, maybe Minidisc will eventually be replaced by some form of Network player which offers equivalent on the spot editing versatility and sound quality.

My first question of the OP's post is : What do you want to get out of your testing experience, and do you think A/B testing is suitable for those goals?

Our whole concept of tone balance is tied to volume. Change the volume, our perceived tone balance changes and no amount of listening experience will ever change that.

You seem to be trying to reference Fletcher-Munson equal loudness curves and not quite getting it right. So let me help you understand- https://en.wikipedia.org/wiki/Equal-loudness_contour
That's not tone. Tone is quality, not just volume. Two violins playing the same note at exactly the same volume can have completely different tone. You have confused and conflated these two different concepts. 

Increasing volume also brings out detail that can be lost at lower volumes again changing our perception and appreciation of the music.

No, volume alone can do nothing to "bring out details". Its true we can't hear anything at all if the volume is too low. But its also true high volume actually hampers hearing and obscures detail. You made a blanket statement that sounds superficially satisfying but has no support in reality. 

Nothing in listening experience will change that either.

Actually the more aware you are of these things the better position you are in to evaluate what you are hearing. In other words it makes you a better listener. Also if nothing in listening experience can change anything then what is the point? There's nothing to learn, and no reason to bother with experience. Another statement it probably felt good to write until its shown to be total malarkey. 

That post was ignorant level 1.

Pretty much. Only you realize too late we are talking about yours, not mine.

Simply putting a tube buffer with a pot in a tape monitor circuit is revealing how little most of this stuff matters to "sound.."  In the end it's not "listening" so much as toe tapping. Like Darren Myers sez. Involvement with the music is the only objective.

cd318
Surprised that you can’t hear much of a difference between SACD and mp3.  I listen primarily to Classical and the differences between mp3 and Redbook are pretty overwhelming, let alone mp3 and SACD.  I do notice that when I play pop mp3 sounds pretty decent.  In fact I don’t own any pop recordings, I just Bluetooth them from my phone via Qobuz.

It’s hard to do truly blind listening.  I have two SACD players in my system, and Oppo 105 and a Sony 5400 ES.  I have a Bryston DAC3 with 4 HDMI inputs, and both players output DSD over HDMI.  I have several SACDs where I have wound up with multiple copies, and recently spent some time comparing the two players, switching between HDMI inputs on the DAC.  Not truly a blind test, and the HDMI hand off isn’t instantaneous either.  I have also switched between one of the HDMI inputs and the Coax input that I use for the streamer and the usb for the PC, comparing rips of CDs, and again it isn’t blind.  I enlisted my wife to change inputs at one point, and she was nice enough to cooperate, but she was clearly bored, and it also led to the question on her part “Since they all sound the same to me,why do you need all of these different boxes?”  so heed my experience and don’t go there

Wish there was better info on the inner workings of the expensive audio boxes.  Some of the info out there is helpful.  Specs are plentiful but really are not helpful but needed.

of course the quality of workmanship may not be of value as it may not matter much in some areas within the box.  Although there are basics that need to be better emphasized by the quality providers.  This great work is being shortchanged by the marketeers, who should be working closely with the designers and creators.  So much to offer about a product that never reaches the buyer.  

That post was ignorant level 1. Our whole concept of tone balance is tied to volume. Change the volume, our perceived tone balance changes and no amount of listening experience will ever change that. Increasing volume also brings out detail that can be lost at lower volumes again changing our perception and appreciation of the music. Nothing in listening experience will change that either.

Lovely thoughts

You guys are talking about methods of switching. But guess what? You can compare two things (A and B) even if it means disconnecting speakers, moving components around. 

Its a comparison, which includes volume and material. If the switching has to be instant, with levels matched and the same material, well then check out my recent post on listening skills. You're beginner Level One: volume. 

At this level I'm not surprised dealers don't want to waste their time. Just ask the guy to hook something up. Listen. Pay attention. You should be able to get through this level in 6 months to a year.

Fair amount of experience but almost impossible to do with speakers. If you can't match levels almost perfect it's meaningless. That is very difficult with speakers making true AB difficult. Even comparing DACs I would be surprised the levels are well matched unless you had a mechanism for it.

I’ve tried it with a Sony MP3 player and a Marantz CD player. I could hear no practical difference and felt a little unwell afterwards. Everything I knew told me there should be a difference, but I didn’t hear one. End result was that I moved largely into the digital is digital camp.

There may be differences but they are from obvious.

I’ve also tried it with different masterings of the same tracks and albums.

Most of the initial differences tend to vanish once the levels are adjusted to match (Mp3Gain etc).

Sometimes though, it’s worth the effort as one track will demonstrate clear differences in dynamics and zest.

Recently I have been comparing different masterings of Adam Faith’s EMI recordings (orchestra/ strings) from that wonderful but often neglected period of musical history - after early Elvis and pre Beatles. I found some significant differences which suggest that slightly different generation tapes may have been used to compile different CDs.

I’ve noticed that, rather bizarrely, as well as volume levels, even differences in CD art can make you decide in favour of one over the other.

In fact it can be so disconcerting that it’s better to use the same art or just turn off the monitor.

So I usually let the tracks play randomly via Windows Media Player without knowing which is which. If there is a meaningful difference it will become soon become apparent.

About 12 years ago I tried it with different brands of CDRs against the original disc. I did find that some brands were a touch closer to the original (the ASDA/ Walmart seemed to be an exact copy) than others but it wasn’t an exhaustive test.

I did something similar with burn speeds and thought that the 4/8 times copy was superior to the 48x copy until I forgot which one I was playing and then realised once again, there was no practical difference.

I would have thought that the slower speed copies would have less errors and they would be audible, but without a way of checking, who knows which copy has less errors? Or at which burn speed.

Blind a/b would be great way to level the playing field if dealers would let you compare speakers, amps etc in this way. Might be embarrassing too.

Can you imagine if you ended up preferring a Rega Planar 3 over a Planar 10?