DAC Measurements vs Actual Experience


I have spent the last two days evaluating which of these two DACs I will be keeping:
1) Benchmark DAC3
2) PS Audio DirectStream

I'm foregoing the use of a dedicated preamp; the chosen DAC will serve that purpose.* 

I had read up on the technical side of each of these exhaustively before purchase. For those of you who aren't familiar, the Benchmark DAC3 measures quite well in every category. The PS Audio DirectStream, on the other hand, not so much. In fact, Audio Science Review places these two near opposite ends of its SINAD (signal-to-noise-and-distortion) list with the PS Audio being positioned near the bottom. Stereophile also provided measurements in their review that painted the PS Audio in an unfavorable light. 

And yet the DirectStream is proving to provide a sound that is more, dare I say, musical. It's a bit more relaxed, sounding a tad reserved in comparison to the DAC3, and the presentation of 100 Hz and lower seems to be slightly vague, but it's also less edgy, possessing a better soundstage, and is overall ever-so-slightly more enjoyable. So what am I doing wrong?

* Is it worth considering putting a dedicated preamp downstream of the DAC3 in hopes of gaining a more favorable sound? 
sixfour3
sixfour3

... the DirectStream is proving to provide a sound that is more ... musical ... relaxed ... less edgy, possessing a better soundstage, and is overall ever-so-slightly more enjoyable. So what am I doing wrong?
I don't think you're doing anything wrong.
In my very personal experience (with top ML components, and X350.5 PASS) the answer is a straight YES. 
A good preamp WILL make a difference, and the better measuring DAC might actually come to the fore!
Seriously, 
Michélle 🇿🇦 
No I wouldn't think putting a preamp after the Benchmark would matter unless it's a tube preamp. Both DACs are performing as expected IMO. 
If any of either if these DACs uses bit-stripping for volume control, a pre WILL make a difference for sure!

The ML390S processor e.g. uses resitor ladders in the analogue domain for volume control, and STILL, going straight (circumventing the volume control) into the ML326S pre, just sounds more 'normal' (less hyped up digital). 

Unless maybe you are considering using a top dCS type DAC straight into your amp, a good pre, and even a solid state one, will make for a more natural sound quality. 

Test it out for yourself and you'll hear.
I 'm not the only one that made that experience either.

That thread by Whitecamaros and his ongoing amp testing endeavours, came to a similar conclusion quite more recently, if you care to read through the tail end of his very long thread, - and incidentally so did John Atkinson of Stereophile some time back as well

So, listen and find out for yourself - since there is disagreement already on this issue. 
Michélle 🇿🇦 

Big difference between accurate and personal preference. The Benchmark may be a far more accurate DAC, but that does not mean for you it is the better DAC.  If you have some ability to set absolute volume though, try to listen at exactly the same volume. One thing you can be confident in, you are not doing anything wrong.
One particular way in which the PS Audio DAC purportedly falls down is the level of noise, possibly due in some way to the use of a transistor output. I confess that I'm not well versed in circuit topology and the idiosyncrasies of certain configurations, but the reviewer from Audio Science Review speculated that that could be the cause.

Another shortcoming is the somewhat limited dynamic range as compared to the Benchmark. I was especially concerned about this given that I have inefficient (86dB/watt/m) speakers. If my logic is correct, a diminished dynamic range with those speakers might make it more difficult to distinguish transients and also some of the more subtle inflections in music. So far, this hasn't borne itself out in my real-world tests. But that's one way in which I'm doubting my own hearing. *Should* I be able to tell a difference here between DACs? 

The level of detail that each DAC is able to extract from the program material, at least as far as I can discern, is virtually identical. The female vocals in the final 2 minutes of Roger Waters' "Amused to Death" has been my go-to challenge for a component's ability to convey delicate nuance. Neither DAC has demonstrated any apparent advantage. 

As far as stepped volume control goes, the DirectStream touts a "bit perfect" digital method. I'll take their word for it. I'm not sure what the Benchmark's configuration is, but I cannot detect any deterioration of performance at any of the levels I've tried (all further clockwise than the 12 o'clock position). I'm evaluating each at the same volume, or as close to the same volume as possible, such that I cannot tell the difference. I'm sitting in a quasi-near-field position relative to the speakers (~6 feet away), mostly using a volume where the system is conveying energy convincingly, but falls well short of concert-level volume. Evaluations have also been made at more "whisper" levels with volume control being done using my streaming device.  

The differences that I'm honing in on have more to do with grain, soundstage and the admittedly vague descriptor of "musicality" than anything else. But I have this nagging feeling that I need to be listening for something more; something that enables me to look at my experience critically compared to the measurements and say, "Ah, yes. There's where my perception is supported by the data." And yet, that's not happening.

These two units each bring a very strong game. I would probably be quite happy with either of them. And yet, I don't want to get this decision wrong. Maybe what I'm asking for here is a "listening lesson" from experienced audiophiles. But to the larger issue, do perceptions and measurements often disagree? 
Yes perception and measurements can disagree. I'm a measurements guy and I also like accuracy, I had a Benchmark DAC3 and liked it I found it very "musical ". The only reason I sold it is I got a new integrated amp with room correction and I really couldn't tell a difference between it and the Benchmark. The PS Audio as you mentioned doesn't measure well it has more noise in a way it is more like vinyl or what a lot of audiophiles say warm, analog like, natural etc.. which isn't uncommon for poor measuring DACs. Some prefer that sound, it's more what we grew up with and associate with sounding right. I will probably get jumped on with what I say they often do, there is nothing wrong with liking what you like or preferring and following your perceptions instead of the measurements don't get hung up on how it measures. Another thing you could try is listen with your ears and not your eyes. Try to get someone to help out and listen to the DACs blind it might surprise you. 
Yes, of course, perception and measurement can, and in many situation, actually WILL,
 disagree. WHY?

In my experience it is something to take as a ’given’, something to accept - not easy actually.
Something even more hard to learn is, the level of ’SQ limitation’ of one’s front-end feeding one’s speakers - THIS includes your cable/system synergies, big time!

Something I had to learn quite recently after over 25 years of looking - in the wrong places too.

In my case, I had taken for granted that my Transparent Ultra, pre to amp, was ’beyond reproach’.
Turns out that was not quite so!

A more like accidental change, of this item (Transparent), for a Madrigal HPC XLR (25 years old!) affected my digital (and analogue) performance most profoundly!

Former harsh like digital sounds, trying to mellow down (make mor 'musical') by e.g. CDP interconnects, using another DAC, with very pricy Transparent Reference digital interconnect, and on - all was of rather marginal outcome.

The ’right’ pre to amp interconnect, all of the sudden, ’changed’ digital harshness into... previously unheard DETAIL, AIR, TONALITY, CLARITY etc.! 

So, the looking for more ’warmth’ in digital reproduction, in this case turned out to be just a band-aid for low cable/system synergy.
This is just an example for what can happen if one ’looks’ in the wrong place - based on some long held assumptions.

I hope this makes some sense, as its not a very easy thing to get accross.
Michélle 🇿🇦

Thank you all for your insightful responses! 

The synergy between components to one another and the interconnects between them, as mentioned in Michélle's response, is something for which I have great respect. Sadly, it's often blind luck that ends up fixing the problem for me. At least in this case I have an apple to compare to another apple, and I *know* the specific component in the system that's responsible for the change in sound. My collection of audiophile quality cables is small (limited to the Nordost Blue Heavens that I'm currently using, a few offering from Tributaries that have never really sounded good to me, and a few sets of Mogami Gold that mostly see use in the music studio). I feel that, while adding cables to the comparison might not be useful at this point, I might swap them out later in this process and see what the results are. 

The sound of the PS Audio is not what I'd call vinyl-like. And I hesitate to describe it as warm. It definitely has the flavor of a solid state device, but manages not to be "out front" nearly as much as the Benchmark. Again, the level of noise isn't perceptibly different than that of the Benchmark in my current system, which is to say that I cannot detect any noise whatsoever in either DAC. 

And with that in mind, I've decided to swap out the Magnepans for Klipsch LaScalas and see if an extremely efficient horn speaker can shed any light on the noise and dynamic range issues. Or, at the very least, see if any differences are revealed between the two DACs in those specific categories while using the other speakers. Who knows? Maybe that will bridge the gap between the measurements and my perception, even if I may actually be getting to the point (albeit slowly) where I no longer put a premium on measurements in choosing audio components. 
I have owned the DAC3 HGC + AHB2 and now the HPA4 preamp + DAC3B. The HPA4 is incredible and makes the DAC3B sound great. It sounds better than the DAC3 HGC. This preamp is so good that I am buying another LA4 (no headphone) for my downstairs system to be paired with some unknown amp.

My upstairs system will be HPA4 + DAC3B + CODA #16.At one time I I was thinking of spending more on a DAC, but now that ship has sailed. I am very happy with this combo.

Get a 30 day trial of the LA4 or HPA4 and be blown away. I took a chance and bought the HPA4 even though I hated headphones. I bought the Meze Empyprean headphones to pair with the HPA4 and DAC3B. It has become a problem every night for me in that I keep saying let’s play one more album, even at 1AM. Just an amazing headphone system I lucked into.

I should add that the HPA4 + Meze have become a gateway headphone drug for me. I just bought a Bryston BHA-1 headphone amp + preamp for a second bedroom. I would have bought another HPA4 for this system but I got the BHA-1 for 1/3 the price. I have not heard it yet, being delivered next week.

BTW - I got the DAC3B for $1560 from a dealer. No 30 day trial for me, since I had prior experience with the DAC3 HGC.

BTW2- If anyone is interested in the Meze. The stock cable is awful compared to the $400 WyWire Platinum XLR I got for it.
Again, the level of noise isn't perceptibly different than that of the Benchmark in my current system
If that were the case they would sound the same since the Benchmark is considered transparent. The "noise" would not be perceived as noise like hiss or hum  but a certain sound like relaxed, reserved, laid back, limited dynamic range. 
I’ve owned the Benchmark it’s brutally honest if the source is lousy, harsh, bright then that’s what you get . If the source is great then it gives you great. There’s no middle ground with Benchmark that’s the way it’s designed and marketed. That’s what I loved about it and it’s what I still get since the DAC in my integrated is transparent. PS Audio is on the other end of the spectrum as you've pointed out. 
Again, the level of noise isn't perceptibly different than that of the Benchmark in my current system
"If that were the case they would sound the same since the Benchmark is considered transparent. The "noise" would not be perceived as noise like hiss or hum but a certain sound like relaxed, reserved, laid back, limited dynamic range."

This may very well be one of those fundamental pieces of information that I hadn't yet grasped. So noise, at least at levels in the -70dB-ish range, is responsible for the relaxed, laid back nature of the sound? And the somewhat stunted dynamic range comes across as a sound that is "less forward."

That's great information. I'm still curious about the imaging, however. Does this also somehow contribute to what I perceive to be a more detailed soundstage? 

Thanks very much for this! 
The type of distortion I'm talking about limits the dynamic range making the bass sound a little soft and highs exaggerated. I'm not sure how that translates to more detail.  Maybe you're more sensitive to middle frequencies and are concentrated there try listening to female vocals and see if the high notes sound like she has a lisp. 
Don't think of this as criticism of the Direct Stream while it measures lousy a lot of people like the sound which is what we're looking for. Some don't like anaylitical and the DS is definitely not that. 
Post removed 
If I had dirty power I would hear noise on all recordings not just lousy mastered ones. I see no need to come on this thread just to insult me and offer nothing to the OP. Thanks for playing.
@djones51, I'm glad you brought up the issue of the source, too. Having little to no information about it, I've considered the 2014 Apple MacBook I'm using to feed the DACs as suspect. I use an AudioQuest Cinnamon USB cable as the interconnect. I stream using Tidal and play my collection of FLAC files using VOX. Could either the computer or the apps be another source of grain and/or edgy-ness? Of particular note, the output volume of each app is set to around the 40%-50% mark in order to be able to turn the DACs up to above the 12 o'clock position. Is this considered bad form? 

I've also heard that dirty power can be a limiting factor for DACs in particular, but for me it's just conjecture at this point. I have no real-world experience regenerating power in any of my systems to know definitively whether or not it's really beneficial, and if so, to what degree. I'm willing to take the word(s) of those who have gone to that length, however, with only the tiniest of grains of salt. 

I apologize to those of you following along if this thread is turning into a free-for-all of topics. But at the same time, so many of these issues are related, so please mention anything that is relevant to the core issue of these two DACs. Thanks! 
RE: "the 12 o'clock position of the volume control"

As the PS Audio DirectStream has no volume knob, let's call it 70%-90% of full volume for both DACs. 
The Benchmark has excellent USB implementation I don't think using the Mac would affect it not sure about the other DAC. If you had a ground loop or mains hum it should be pretty apparent.  Dirty power seems to be the most recent craze. Some might have problems with their power I've never noticed any. 
I've never liked using volume in apps I set it to fixed and use the controls on the preamp or DAC or integrated depending on what I'm using. Obviously you can't do that with the DS. 
Post removed 
Hello,  I have owned the PS Audio DS DAC with Bridge, the Auralic Vega G2 and the Mola Mola Tambaqui (in the Makua Pre amp).  Many good posts here.  My experiences:

1) +1 to Michelle:  in all situations a good pre amp improved everything.  It just did.  I tried both the PS Audio and the Auralic without and with different pre amps (PS Audio BHK Signature and Audionet Pre 1 G3).

2) I identify with your comments about the sound of the PS Audio; In comparison it was veiled and dirty sounding to me.  Also there was always noise with the PS Audio stuff.  The Auralic is way more clear and detailed.  the Tambaqui is amazingly revealing and wonderful.

3) +1 to Michelle again!  The cables and interconnects make a huge difference.  With the PS Audio stuff I upgraded to some mid-grade silver clad balanced interconnects and speaker cables.  They helped the PS Audio equipment gain clarity and resolution.  But with the more resolving DAC's, the silver clad cables were too harsh.  I demo's several brands and chose Audience AU24-SX.  Smooooooth all copper, great midrange and bass, clear but not harsh. Relaxing in comparison.  

4) Get the Mac out of the system...all my DAC's also streamed so easy there, no separate streamer required.
My journey took me to clarity and resolution...hearing things I never heard before.  Good Luck!  Ken
If you want a shrill/way too analytical/fatigue sounding dac, the benchmark is your dac. I would walk into a room at an audio show and know if it was a benchmark dac, I could only stand listening for a couple minutes before we had to leave.If you want a musical sounding/fpga based/networked attached Roon endpoint, get the ps audio. A couple times a year you get a free software upgrade that has improved the sound each release as well as adding functionality like MQA. Measurements don’t tell the whole story, your ears have the final word
Many DACS measure like crap but sound great.  Forget measurements and go with what your ears say and not worry about it.
measurements can be useful when designing gear, but they are almost useless to determine whether a piece of gear provides musical enjoyment

accuracy is the signal as a proxy for what sounds real and what sounds good is mightily flawed as a concept - we are talking about REPRODUCTION of music here... a watercolor, a photograph of an event ... the distance from the real thing is huge

furthermore, for produced/mixed music, what is real and accurate anyway? it just matters if things sound good, as perceived by the listener

"he died of a heart attack!!" - but why?  his body temperature was exactly 98.6*!!!!!!!!!!!
OMG!
Do you actually think that "lousy, harsh, bright" would be allowed on a recording:

- By the recording engineer?

- By the mastering engineer?

- By the record producer?

- By the record label?

- By the recording artist?

Hmmm. No way. Ain’t gonna happen.



Listen to RUSH "Vapor Trails" before they re-did that disc. I actually threw the first one in the garbage.

You’re not doing anything wrong. You discovered that good measurements are not a guarantee for good sound.
Post removed 
A sincere thank you to all of you who have contributed meaningful, well thought-out arguments and information in regards to this issue. 
I suppose I now consider myself a convert. Measurements alone, while still being relevant to the discipline of design and perhaps initial consideration for potential buyers, do not, cannot account for the subjectivity in how we perceive, and thus enjoy, our experiences with our systems. 
It is perhaps commentary on our own egos that some of us (myself included) want our perceptions to align perfectly with what specifications say is the ideal. And yet, it turns out that that ideal is not necessarily what gets my toe tapping. 
Kudos to jjss49 for providing what is perhaps the best analogies I’ve yet heard on this subject. 
Special thanks to djones51 for shedding light on technical issues, specifically how we perceive noise, which really set the stage for me to come to terms with the concept that my preferences don’t necessarily line up with the measured ideal. 
Thanks also go to everyone else whose simple advice to trust my own ears prodded me to do just that. 
I have a few more considerations to weigh, mostly having to do with price and performance value, but it looks likely that the PS Audio DirectStream just found its new home. 
@sixfour3,

As someone who believes in science, measurements, and subjective listening, I believe there are some serious misconceptions about "measurements", and the position that those who use measurements take, and what they mean. That misconception comes from blind attachment to absolute measurements (a well known audiosciencereview person), as well as those on the other side who don’t don’t understand measurements or want to sweep them under a rug in all cases.

Measurements will never indicate individual preference. They may indicate, after large scale tests like Harmon/Toole did, that most people will prefer a particular set of measured values, but they will never guarantee what any individual will prefer.

Measurements may also reveal "uncomfortable" truths, like high resolution digital does have more "information" and can capture and play back what comes off that microphone more accurately than analog tape and vinyl. More accurately does not mean you will prefer it, and because of side effects of vinyl, there may be psycho-acoustic effects that allow more of certain types of information to reach your brain. Strangely enough, more noise can also allow you to detect lower level signals in some cases. Reduced information can allow you to focus on that which is most important which may be lost in a higher information "stream".

And just to "tweak" the never measurements crowd :-), while measurements cannot reveal what we will subjectively like, they are pretty good at identifying what can or cannot make audible differences, but if you are unsure, you can always perform an unbiased test with your ears/brain :-)



@roberttdid This is some amazing writing. I am going to email your post for future reference.
The psaudio has received many great reviews from pro to regular customers regardless of the measurements.
 I have one and love it even though I know there's dacs that measure better.
I've never heard a Ess based dac that sounded as natural and musical. 
When I can upgrade it will probably be for a chord, dcs or msb and not a standard chip dac. 

The directstream does also sound better with a preamp. 
The suite of measurements ASR runs through is meaningless in terms of making any purchase decision - granted it is one data point. Benchmark and ADSM both measure well, but ADSM sounds amazing and analogue but Benchmark sounds like POS.
So by that logic I can assume the ADSM sounds like a POS. Different strokes for different folks.
Analogue ... What exactly does that mean? Pretty much you are right YYZ
I guess the ADSM is the Linn ? I looked on the ASR site and see the Linn and Benchmark are very close on their chart. I've heard the Benchmark but not the Linn and at the price I doubt I ever will. My guess is in a blind test I doubt I could tell one from the other. Once DACs have reached a level where their performance is beyond human audibly I don't understand how they could be told apart. I have listened to other DACs in the same measurement range as these two and I couldn't. I know I'm deaf and my system is garbage. 
djones51,

Some DACs, even high end ones are intentionally colored. Some claim that a minimum phase output filter improves the sound, but it is still a coloration and not true to the data. It sounds different and you may like that and you may not. Some like MQA, some think it adds an unnatural quality to instruments, even if some find it pleasant.
I have heard a few recent Linn’s, including the Selekt DSM. A little veiled compared to my preferences but not bad. I was tapping my toes with them, but I do that for all good music.

Just to show a little more enthusiasm for my HPA4 + DAC3B combo I want to say that I have been listening to the headphone section of this preamp and it sounds great. The real fun part for me is hearing so much new musical passages in well know material, it is shocking. For example, I was playing Stairway to Heaven the other night and I heard John Bonhoim (sp?) play some drum track that I had never heard before. I am hearing stuff like this on many well known recordings. This maybe not what everyone wants in their music, but for me it makes me feel like I am in the studio with the performers.

I actually had a similar reaction with the Benchmark DAC3 HGC + AHB2 amp. It was not as good as what I hear now with the HPA4 preamp but that THX circuit on the Benchmark really works for me.

I need a new amp to make my speakers sound as good as the headphones, likely the CODA #16 and/or the AHB2.
I suppose we all prefer different sound. I owned Benchmark at the same time as older Linn Klimax. I got rid of Benchmark because it sounded like POS next to Linn. 
You do you and carry on with measurement first mentality. I use my ears and keep the equipment that sounds good to me. 
Hey Chesebert can you tell me who wrote this in 2011 and whose fairly limited posts have been almost all about DACs and a large portion about Linn DACs?  Biased?


I personally don't mind listening to mp3 (320kps) using Klimax DS. It sounds a whole lota more analog and musical than the same CD played back on the CDP that Klimax replaced. Yes, mp3s sound great, CDs sound amazing and hi-res sounds heavenly.

Contrary to what this implies these two concepts are not mutually exclusive. 


You do you and carry on with measurement first mentality. I use my ears and keep the equipment that sounds good to me.
@rbstehno I don’t understand what you are trying to imply here. I write about stuff I own and heard. I do have a lot of experience with Linn DS as I have owned various DS over a decade. I don’t talk much about my Ayre and Chord front end because they don’t sound as good as top of the line Linn DS. 
Re the Klimax in 2011, it was a true revelation back then. Playing 320k mp3 from Linn Radio was a treat. 

@djones51 To me “measurement first” subscribers will keep a piece of gear solely because of measurement even if this gear is steering the person to listen to certain kind of music. “Listening first” will get rid of a piece if it sounds bad to that person even though the gear measures well. 
I believe “measurement first” and “listening first” are mutually exclusive. 
You give two different parameters. I look at measurements first if it sounds bad I don't buy it or keep it. The type of music I listen to has nothing to do with the components I buy. A DAC doesn't steer me to listen to certain kinds of music. 
Looks like we are all in agreement here. I approach buying gear in pretty much the same way. I eliminate truly horrendous measuring gear (like border patrol) and then see what’s available near me for audition. 
Sixfour3, 
You have come to the point of recognizing that what measures well doesn't ensure sounding well. Conversely sounding well doesn't necessarily correlate with measuring well. IMHO listeners are in the long run much better off choosing audio components that sound best to 'their own ears. It doesn't matter what others may think or hear differently than you. It's purely subjective.  Just be happy and select what you determine to be the better sounding more satisfying component. 

As this thread has clearly demonstrated people are simply going to disagree on how a particular audio component sounds. Some posters here find Benchmark to sound excellent and others aren't impressed with them by any measure. In my own encounters with this brand's products they struck me as clinical/analytical/sterile. If you find the P.S,Audio DAC more enjoyable and engaging,  trust your own ears. I wouldn't obsess over the measurements at all.
Charles 

Many issued recordings are flawed in some way or other.  Perhaps most are.  If you are a certain kind of purist who wants to hear exactly what the recording as issued is presenting, then by all means get the most "accurate" components possible, which is something a lot easier to measure with DACs than, say, loudspeakers.  If you want to sit entranced by most if not all of your collection, you may prefer something else.

For instance, I remember the early Deutsche Grammophon recordings that were recorded digitally and issued DDD on CD were usually pretty much unlistenable.  Things have improved since then, but still...

Is orange a "distortion" of yellow?  If you happen to like the color orange over the color yellow, is that wrong?

I must say that I have no clue how to interpret the measurements on the HPA4 and DAC3B combo.  I just listened, especially to the silence.

The best ESS based chip DACs I have heard were not the Benchmarks but the $6K Mirrus DAC from these guys, who now seem out of business.

https://resonessencelabs.com/

To my ears, better than the Benchmark and also any of the Linn stuff. I apply the concept of Law of Diminishing returns to DACs over any other audio component. For $1560 new that the DAC3B cost me I am consider it a big win.

I have heard a lot of DACs. I am not saying my choice is going to be great for anyone else just saying it is amazing fro me. 
But even your ears can trick you. An audiophile bought a pair of well reviewed speakers and loved them at the store and initially when he got them home. But over time he just felt there was something "wrong". Some music sounded amazing, some not so much. I was able to tell him in 15 minutes with a laptop and microphone that the crossover frequency was too high, hurting dispersion and it looked like some cone breakup. Sounded awesome on vocals, guitar, even most pianos as it was in the sweet spot of what was a very good driver, but used wrong at a system level.