Best practices when conducting a DAC comparison

Why would anybody buy a dac without MQA support? Just a few years ago, you probably had the same naysayers about DSD as you do about MQA. A good DSD cut is far superior than the same 16/44 cut. If you purchase an expensive dac today without MQA support nor the opportunity for a future upgrade, your dac will be a paperweight in just a short period of time. So not only will you be wasting your $, you will be missing out on hearing some great music!

Why would anybody buy a dac without MQA support? Just a few years ago, you probably had the same naysayers about DSD as you do about MQA. A good DSD cut is far superior than the same 16/44 cut. If you purchase an expensive dac today without MQA support nor the opportunity for a future upgrade, your dac will be a paperweight in just a short period of time. So not only will you be wasting your $, you will be missing out on hearing some great music!

As far as I know, MQA uses a *lossy* compression codec. I have absolutely no desire for a lossy format, in particular because the live music I collect and listen to is *lossless*. So this is why I chose a DAC (Schiit Yggdrasil) without MQA support.

@bigkidz Thanks for your response and the details shared. I generally agree that if one doesn’t pick out differences immediately then it can be a toss up choice (having controlled and equalized the variables as best as possible).

I find this also applies to out of the box performance. If one can pick up differences at that initial point, then they become clearer as a component gets further run-in time on it. This has recently been the case with a couple of components.

I have also had the opposite experiences.

For example, I had to spend much more time A / B -ing to drill out and appreciate differences with a recent USB cable comparison. It was tough and time consuming but well worth it and very educational for me.

In some cases (where there were readily apparent differences upfront) elucidating whether they met my goals and preferences was a drawn out process. This happened with a recent speaker comparison.

I believe our level of listening experience and development of ’hearing’ expertise comes into play as well. Some may be much further along the spectrum, which would make elucidating the differences easier and quicker.

@georgehifi   Is there a 'cheater' way to get close enough? : )

I can do so, if you and others believe it is critically important.

@rbstehno   I have a long history of audio bricks as doorstops. : )

It would have been nice to have. In the end it wasn't a deal breaker for me.

@georgehifi Is there a ’cheater’ way to get close enough? : )

No.
If you do it by ear, you could be as much as 3db out on level matching.
Cheers George

@david_ten 

As I mentioned in my 11-06-2017 1:17pm post, "I ensured the volume control on my amp never changed". So while I do not disagree with George, it could be a moot point and you may want to forgo the hassle of precise level matching. Why, because the difference among the DACs may (and likely should or will) be audible from a tonality, dimensionality, or other characteristic that can be readily detected irrespective of level matching. Personally I'd "worry" about precise level matching only when and if you're at the stage when all else is equal. 

"I ensured the volume control on my amp never changed"
  it could be a moot point and you may want to forgo the hassle of precise level matching.

All dacs have different analogue outputs levels even if it's by a 1/10th of a volt, how can anyone compare dacs sound, if the levels are not matched, even a minute increase in level will favour that dac in an A/B comparision.

Cheers George  

All dacs have different analogue outputs levels even if it’s by a 1/10th of a volt, how can anyone compare dacs sound, if the levels are not matched, even a minute increase in level will favour that dac in an A/B comparision.

Hi George. You are right. I have no disagreement with regard to the different output voltages. In my case, I compared the maximum output specification of the Yggdrasil (4.0V RMS (balanced), 2.0V RMS (single-ended)) and that of the Oppo UDP205 (RCA) 2.1±0.2Vrms. (XLR) 4.2±0.4Vrms) and concluded that for my purposes the output voltages were close enough to not be a concern.

I knew my comparison would be less than scientifically exactly perfect, however, I did want/desire as accurate results as possible *and practical*. Practical being the operative word, and precise level matching may not be practical or necessary.

I disagree that a minute increase in level well *necessarily* favor a DAC in a A/B comparison. It may, or it may not. What I’m merely suggesting to the OP is that there are other factors besides "voltage output levels" that matter even more when doing the A/B. For instance, the Oppo treble is more "tinny" than that of the Yggdrasil. That is true regardless of the output voltage, and one doesn’t need to jump through hoops and level match to find out.

Obviously one does not want to listen to DAC "A" at 60db and DAC "B" at 100db. Sure, if the *result* of the output levels produce egregiously different volume then something is amiss and that could/would invalidate a test. But a 10th of a volt, all else being equal, I think not.

EDIT:

Should the OP have the proper equipment to precisely level match the output voltage, certainly there is nothing wrong in that and in fact that would be preferred. So in no way am I suggesting not to level match. Instead, I'm calling attention to the merit of doing so relative to other items and the likely hassle involved in attempting to *precisely* match.

@georgehifi  and @gdhal  Thanks for your thoughts on the level matching point you are making and are mainly in agreement on. 

@stfoth  I like the 'enjoyment' part of your assessment. Your primary suggestion on how to go about the evaluation is valid and has good company in that it has also been offered up (similarly) as an approach by a couple of others. 

My way to audition a new DAC is with music (a) that I am very familiar with (b) with a mixture of recordings that I love and hate on the existing DAC. Recordings one loves should sound equally as good or better with a new DAC. Just as important is when recordings that grated on you with the old DAC no longer offend and become interesting to you ; these are signs you have fixed a broken area in the old DAC. The pop genre tends to be sprinkled with recordings which excite bad sound on lesser DAC's.

Most DAC improvement is in increased detail   and reduced digital artifacts. DAC's can have tone color signatures (lean/lush/neutral) which will also affect your preference. DAC's also cannot produce what doesn't reach them correctly, anemic bass and blurry detail for accompanying instruments can be signs of jitter issues with the source solution feeding the DAC.

I have MQA in my beloved DAC, and it's disabled. I find I prefer high resolution with a different filter than it demands. I don't hear a benefit of MQA. Sorry.

As for auditioning, make sure both components are 24 hours warm and broken in. YMMV, but that's a more fair test. Ice cold, some DAC's do better than others.

Next, compare red book as much as high resolution. Don't assume that the DAC which has the most difference is the better one. The last generation of DAC's play red book exceedingly well. The previous generations only played high rez well.

Best,

E

I agree with bigkdz.  If you can't tell which one is better within the first 30 seconds there really isn't a significant difference.

Level matching may make a fair comparison if you are switching back and forth, but if at that point you are straining to hear a difference you're wasting time.

Another approach would be to play a familiar cut for a short time at different volume levels, then do the same for the competition.

....If you can't tell which one is better within the first 30 seconds there really isn't a significant difference....

Nor should there be if you already have a highly-decent sounding system.

I agree with bigkdz. If you can’t tell which one is better within the first 30 seconds there really isn’t a significant difference.

What????? Jesus. What exact test tones are you using? It takes several tracks just to fully exercise a DAC. Bass, air, imaging, impact which is not to mention some issues like fatigue, take much longer than 30 seconds to set in. Then how will you compare the Redbook, High Rez and DSD performance? 30 seconds is barely long enough to eliminate a DAC you will never like, assuming it is warmed up but is not a fair way to choose among good performers, unless you believe they all sound the same, and specs are all that matter. If this is you, buy the cheapest you can find.

Best,

E

Erik,
I do not listen to test tones or specs.  I only use red book as digital source.  There are few innovations in the digital conversion world that truly separate one product from another.  Most use the same chip sets with various number of chips deployed and specific parts implemented in the circuit.  

I have four four digital sources available in my main system.  Three of them resemble one another in that they posses a more noticeable tell tail digital signature.  The least desirable to my ears is the Oppo 105D circa  2015.  Next up an early nineties Sony followed by a mid nineties tube dac from VAC.  Then there is one that within that first 30 seconds of playback sounds that much more realistic to me that further comparison is pointless.

 I also have an analog system while not state of the art is far above an entry level setup.  If both the the analog and digital sound more similar than different this is a good thing in my experience.

However, the main goal for both is to sound like real music and convey the emotions and content to make me sit back and get lost in what's playing which is happening now as I type.  Type a bit, stop and listen, then a few more words on the page.

An example of the 30 seconds test is track 2 of Holly Coles  Don't Smoke in Bed.  It's not a complex piece but revealing of digital nasties.

OBTW, I can afford more but what I have pleases me and that is all that I require.  You're invited anytime for a listen.

GeorgeHIFI is so right about volume ,many would choose any higher level even the tiniest amount over the other component ,

.Someone mentioned MQA and why would anyone not choose a dac that didn’t feature this ?  Recording engineer and owner of AIX Records Mark Waldrep including others in the industry have plenty to say about MQA overthe past couple years , RealHD-.com is a great place to start ,

David 10 , have fun with your evaluation,.

First, thanks for the additional thoughts shared via the previous few posts.

Second, I will try to consolidate everyone’s suggestions on approach and put together a summary.

Third, the Denafrips Terminator is broken in and is performing very well. I’ll be swapping in the Schiit Yggdrasil to get some listening time with it (and the new Gen 5 USB board).

Fourth, a major UPDATE:

I thought I’d conduct the comparison over the Thanksgiving Holiday...but other priorities got in the way. I wish I had, because I now have a new element in the system. I’ve been waiting for a speaker delivery, which was planned and scheduled for before the Denafrips DAC was delivered. The speaker was delivered yesterday.

My original plan was to have the new speaker fully broken in as well as to have a significant amount of personal listening time with it. Since I plan to keep the new speaker (and the DAC that performs / has the best synergy with it) I think the best approach is conduct the DAC comparison once the speaker is broken in and I’m familiar with the new speaker.

Thoughts in agreement or contrary to this? If otherwise, I’m interested in how you would proceed? Thanks.

David,
I have now owned and auditioned a Bryston BDA-3, Luxman DA-06, and the Oppo 105d with its internal DAC.  Each, as you know, uses a different DAC chip: respectively AKM, Sabre Burr- Brown.
I now use exclusively the Bryston, sourced via USB by a Bryston BDP-3 streamer, or the Oppo disc player via HDMI.
The characteristics of these DAC's has been described by others, and there may be general agreement that the Luxman is 'very smooth', the Oppo (sabre) detailed but harsh, the Bryston in-between.
My personal impressions:
1. The Luxman employed for chamber music...string quartets, piano trios and quartets...
is 'muddy'....the lack of detail increasingly becomes annoying.  Its employ for solo piano invites similar comment.  Try as I might to persuade myself that the 'more expensive sound'...ie, the Luxman's...should be better than the Oppo's, I had to throw in the towel and
avoid the Luxman for strings and piano.
2. The Luxman employed for voice is a delight.  I've read of various hi-fi components
that a 'smoother' high frequency response decreases listener fatigue.
Such comment applies to the Luxman used, for instance, for listening to an
opera DVD.
3. That virtue is such that even 'downrated' DVD audio sourced to the Luxman via the Oppo's coaxial audio output was superior to the Oppo's internal audio decoding of opera DVD's,
including blu-ray DVD's.
4. The Bryston BDA-3 is a good compromise...' 
I struggled to parse the foregoing impressions in respect of digital source format...standard CD, SACD, hi res PCM, double speed SACD.   The sound quality seems to improve as one progresses from the beginning to the end of that list, but the DAC comparisons...at least my ear...are the same. 

A most excellent list, Steve.

The question that bugs me is that of optimizing each component for best results. Should David use a reclocker for both pieces in the comparison even if one benefits and the other doesn’t (assumedly because the latter has better addressed the issue of noise and jitter reduction internally)? Does that fairly represent the potential user experience of each? What about different cabling/power cords that optimize each independently?

The purists’ "apples to apples" mentality would seem to require that the exact same cabling (even using the included power cords and perhaps generic digital cabling) with no external devices in play. Sounds fair, but what if one component would benefit tremendously from a reclocker or specific cabling to the point that it significantly outperforms the other and/or changes David’s preference? Even worse if one is much less costly than the other and investing a relatively small amount (i.e. that the total investment would be significantly less) would make it equal or even to outperform the much more expensive competitor.

The Terminator lists for ~$4400 and the Yggsrasil for $2400. Would spending less than $2k to optimize the Yggdrasil with a reclocker, cabling, etc. yield a better sonic result, thus making it a better purchase decision for the same or lower investment?

Seems that in the "bang for the buck" approach that the answer would be to address this with a "same or less total cash outlay" comparison as this is the real world dilemma for many of us. But then there are those that will choose to use each without any "supplements" (reclockers or upscale cabling, etc) that would find value only from the "apples to apples" comparison. And also those seeking an "all out assault" that may have interest in an ultimate sound quality comparison with each having every enhancement specifically optimized for that particular component regardless of cost. Who, short of the most diehard professional reviewer, has a sufficient number of these ancillary components available to even begin to satisfy all possible curiosity?

David has a real challenge on his hands trying to potentially satisfy each category of reader. I wish him the best of luck.

Dave

Thanks to everyone for your guidance and to the recent helpful posts from @seventies @audioengr  and @dlcockrum 

I've learned much from each poster.

I finished the comparison between the Schiit Yggdrasil and the Denafrips Terminator this morning. The comparison got held up for a number of reasons but all-in-all the delays, in retrospect, were actually helpful and made for a stronger and more robust evaluation.

First, I was able to get to a place where all the components were well broken in.

Second, I was able to spend significant time with my system and each of the 'newer' components and cabling and I now have a very good feel and sense for them. 

Third, though the comparison is Subjective, I was able to control many of the variables to level the playing field --- as much as I reasonably could; however, the review process will not meet bench level rigor.

Fourth, the added time has allowed me to think through and reflect on the process and what I'm looking to achieve much, much more thoroughly. 

Fifth, I was able to get another audiophile's perspective on where my system is 'performance wise' as well as getting feedback on shortcomings. I also wanted a second opinion on What I 'hear' and How I 'hear' and to discuss my audio preferences 'live' and based on my system. I've also had a number of phone discussions with audiophiles active on Audiogon to answer questions and bounce thoughts and approaches off of.

I will be writing up my findings and takeaways starting tomorrow and should be able to wrap it up over the next few days. Once it's finalized, I will post a link for those who may be interested and curious.

Hey guys, my write up is posted here:

http://terminatorandyggdrasil.blogspot.com/

And here:

https://forum.audiogon.com/discussions/denafrips-terminator-and-schiit-yggdrasil-evaluation-and-comp...

This thread guided my approach, though it is not as rigorous as some suggested or how you might conduct your own evaluations.

The suggestions, however, certainly became part of the process and for that I am thankful and grateful to all who posted. I learned a lot through your feedback. Thank you.