On Saturday, February 24, a few members of the San Diego, Los Angeles and Palm Springs audio communities conducted a blind shoot-out at the home of one of the members of the San Diego Music and Audio Guild. The five CD Players selected for evaluation were: 1) a Resolution Audio Opus 21 (modified by Great Northern Sound), 2) the dcs standalone player, 3) a Meridian 808 Signature, 4) a EMM Labs Signature configuration (CDSD/DCC2 combo), and 5) an APL NWO 2.5T (the 2.5T is a 2.5 featuring a redesigned tube output stage and other improvements).
The ground rules for the shoot-out specified that two randomly draw players would be compared head-to-head, and the winner would then be compared against the next randomly drawn player, until only one unit survived (the so-called King-of-the-Hill method). One of our most knowledgeable members would set up each of the two competing pairs behind a curtain, adjust for volume, etc. and would not participate in the voting. Alex Peychev was the only manufacturer present, and he agreed to express no opinion until the completion of the formal process, and he also did not participate in the voting. The five of us who did the voting did so by an immediate and simultaneous show of hands after each pairing after each selection. Two pieces of well-recorded classical music on Red Book CDs were chosen because they offered a range of instrumental and vocal sonic charactistics. And since each participant voted for each piece separately, there was a total of 10 votes up for grabs at each head-to-head audition. Finally, although we all took informal notes, there was no attempt at detailed analysis recorded -- just the raw vote tally.
And now for the results:
In pairing number 1, the dcs won handily over the modified Opus 21, 9 votes to 1.
In pairing number 2, the dcs again came out on top, this time against the Meridian 808, 9 votes to 1.
In pairing number 3, the Meitner Signature was preferred over the dcs, by a closer but consistent margin (we repeated some of the head-to-head tests at the requests of the participants). The vote was 6 to 4.
Finally, in pairing number 5, the APL 2.5T bested the Meitner, 7 votes to 3.
In the interest of configuration consistance, all these auditions involved the use of a power regenerator supplying power to each of the players and involved going through a pre-amp.
This concluded the blind portion of the shoot-out. All expressed the view that the comparisons had been fairly conducted, and that even though one of the comparisons was close, the rankings overall represented a true consensus of the group's feelings.
Thereafter, without the use blind listening, we tried certain variations at the request of various of the particiapans. These involved the Meitner and the APL units exclusively, and may be summarized as follows:
First, when the APL 2.5T was removed from the power regenerator and plugged into the wall, its performance improved significantly. (Alex attributed this to the fact that the 2.5T features a linear power supply). When the Meitner unit(which utilizes a switching power supply) was plugged into the wall, its sonics deteriorated, and so it was restored to the power regenerator.
Second, when we auditioned a limited number of SACDs, the performance on both units was even better, but the improvement on the APL was unanimously felt to be dramatic. The group concluded we had just experienced "an SACD blowout".
The above concludes the agreed-to results on the blind shoot-out. What follows is an overview of my own personal assessment of the qualitative differences I observed in the top three performers.
First of all the dcs and the Meitner are both clearly state of the art players. That the dcs scored as well as it did in its standalone implementation is in my opinion very significant. And for those of us who have auditioned prior implementations of the Meitner in previous shoot-outs, this unit is truly at the top of its game, and although it was close, had the edge on the dcs. Both the dcs and the Meitner showed all the traits one would expect on a Class A player -- excellent tonality, imaging, soundstaging, bass extension, transparency, resolution, delineation, etc.
But from my point of view, the APL 2.5T had all of the above, plus two deminsions that I feel make it truly unique. First of all, the life-like quality of the tonality across the spectrum was spot-on on all forms of instruments and voice. An second, and more difficult to describe, I had the uncany feeling that I was in the presence of real music -- lots or "air", spatial cues, etc. that simply add up to a sense of realism that I have never experienced before. When I closed my eyes, I truly felt that I was in the room with live music. What can I say.
Obviously, I invite others of the participants to express their views on-line.
Do blind tests really matter in the world of high end audio? I ask because I am certain that many or most Audiophiles would look at results of say a $500 item and a $10,000 item and even if the budget item "trounced" the high ticket item, most would still buy the expensive piece because if it costs more it HAS to be better, face it alot buy with their eyes...not ears. I am also willing to bet MANY items compare very well as super high priced items for a fraction of the cost......and we do not allow ourselves to admit this truth to our inner audio geek. Two case in points, Dave Wilson demo with a cheap Parasound amp, also another with an IPOD...both left the crowd pleased, but how many do you think actually incorporated those items with Wilson speakers?
Intriguing but not entirely relevant. Not description was included in the opening post on associated equipment used, the room, etc. except a little by Alex who mentions Pipedreams (unspecified model), some unknown model of Lamm amps, and some model of PS Audio power regenerator powering who knows what. Never mind the preamp, cabling, etc. But we know 16 year old Lagavulin Scotch was consumed.
I am very appreciative that this comparison was done; it didn't appear like advertising to me (although maybe it would have been "wiser" to have Alex leave for a few hours). I also appreciate all the follow-up questions, and the clarification of other(s) who were there. At the risk of anyone thinking I'm wringing my hands (I'm not - just curious), my question has to do with warm-up of players, potential interaction of having other players plugged in simultaneously, etc. Again, I'm just curious, not suggesting at all that the results of the testing done here was invalid. One of the reasons I ask is I'm very aware in my own system that once a component (esp. digital)is turned off, it can take some time to get back up to optimum sound. Seems like those who did this testing thought of almost everything - and did "due diligence" to try to control all the variables they could. So...just wondering how issues like this were handled. Thanks for doing this comparison and posting the results. This kind of thing - including subsequent discussion - is, in the end, a great service to us all. IMHO p.s. Wouldn't it be great if each of these players (or at least the top three) could be used in each of the 5 individuals' systems for a week or so? I'd love to hear their thoughts after that.
Tbg - The APL unit with its volume control and line input capabilities could replace my current digital player, preamp, 2 PCs and IC. However, I would still need at least another $10k to afford either of the top two players if I were to sell all this gear. This is too expensive for me.
I quickly put the same tracks in my system as soon as I could after the event to get a feel for how my set up fared against what I heard just 30-40 minutes earlier.
It did not take long, about two bars of music on either track, to notice the refinement in transparency and the smooth resolving ability the top three players had compared to my digital player. This was particularly evident in the mid-high to HF range. I have a maxed out Philips SACD1000 that was modded by APL. I did not start out looking for an APL player. In fact I originally wanted the Opus 21 or the Exemplar 3910. The deal on the Philips was too good to pass up so I got it instead. The SACD1000 is quite capable and is competitive in image focus, soundstaging ability, tonal balance, with an engaging combination of mid and midbass articulation and body, and bass extension and dynamics. Unfortunately it just does not have mid-high to HF performance that the top three units had in our blind sessions.
As to my take on the comparisons, the DCS had a more lively/dynamic presentation than either the Meitner or the APL, which is probably why I liked it better than the Meitner in the quieter choral piece. My vote changed for the Meitner when the orchestral track was played. Despite a less deep soundstage compared to the DCS, its smoothness and more natural presentation of instruments won me over. [This is really where the latest wersion of the Meitner really shines compared to what I remember of the previous version.] The DCS and APL presented a better delineated and deeper soundstage than the Meitner. The Meitner and especially the APL were better at presenting more believable natural tonal and harmonic characteristics of voices and instruments.
So wish me luck on my quest to find a more affordable digital player that can come close enough to the DCS, Meitner and APL.
IMO, Just two tracks (and both classical at that) are not enough to claim any digital pairing superior over the other. You need varied kind of music. well recorded acoustic music is preferable of course. PLUS few hours between five units are also not enough to be conclusive. Aural memory ususally is also short lived and you gotta go back and refresh pairing many number of times to get more data samples. The panel also need to have breaks in between to get back bearings/tonal reference. You get kind of tone deaf after a while. I am not saying APL may not be clean winner or others a close second but to be scientifically correct simply more data samples is needed. IMHO.
Tvad, I agree. This is about a close to an interesting test as I can conceive. It just shows how difficult it would be were reviewers to do such a shoot-out.
One additional question for me is whether those participating would sell their unit to buy the winner.
I once participated in a shootout very much like this but comparing preamps. Among the six or seven preamps including those manufactured by some of the participants, the Bozak had the highest rating! I for one, never thought to buy one. I did scratch my head about these results, however.
Really great test and write up. Thanks for taking the time to post the results.
When you were experimenting after the formal testing and came to the conclusion that the NWO 2.5T blew away the Meitner on SACD, was this also true on redbook? Thanks.
Bob_reynolds - Level matching was done for each pairing using an uncorrelated pink noise track from a test disc played through each unit. Measurements were made using a pro spectrum analyzer taking SPL readings from the listening position. The audio chain is as follows: CD, pre, amp, speakers.
I had no intention to belittle the effort or sincerity. Without disclosure of the rest of the system (better but impractical, would have been two or three different systems) and the participants in the test and some idea of their own systems to guage personal taste, the effort was only for themselves.
Anonymous votes can elect a president(not necessarily) but that's not what we're talking about here. The best intentions can be lost in communication.
Interesting that the last to be tested was the preferred. Results might change if the order was different. Psychology always affects perception. Also, once the group decided that the last choice was best, was this player then compared back against all of the previous? If A if preferred to B, B over C and C over D, it is not 100% sure that A will be preferred over D.
The fact that there were audible differences in such high quality players is really scary. It begs the question as to which player was correct? They can't all be right. One has to be in closest conformance to what is on the CD (which sounds nicest with a given set up or speakers is a perceptive evaluation). I assume they all had THD+N to three decimal places or more at that quality level.
Thanks for all the clarifications. As for my questioning, it was NOT to imply any "leading the witness" but rather to help those of us not present to understand the biases and overall sound characteristics that the system is generally producing. In my sytem, for example, you could take a EMM stack at $50k and a tubed Monarchy DAC at $800 and likely the group familiar with my sound would favor the Monarchy DAC...why? Cuz any more detail and speed to my very tipsy (Leaning Tower of Pieza ) analytical-leaning system and the result would likely be perceived as "too much" and tip the scales. However, that's my systems fault, not the EMM gears'. Its potential would remain unrealized. Ted
Tbg - Two of the five voters own APL modded Denon 3910. Alex was there because he brought the NWO 2.5T. But he did not vote, provided no commentaries during the comparisons and was not involved in the set up.
Because of the above, the player that started each pairing was scrambled/varied to as to keep the voters on their toes. This random selection of which player goes first when using either track 1 or track 2 was applied after the first pairing when it was obvious which player was used due to its location on the rack (even though it was covered and the voters could not see it). So after round 1 no one was really sure which player was playing despite where each unit was located on the rack. You can imagine how careful one must be to accomplish this. Thanks to the aid of a couple of blankets and remote controls when available, we were able to. In other words, we loaded both players simultaneously and started and stopped them similarly. Only one player had the CD of interest and was connected to the preamp. The other was connected but was playing a different disc as a "dummy" unit. We even switched the input connections for the players to be sure that there was no consistency in how one player was connected to the preamp. Now because they were kept from being seen, the voters could not see the front panels that would have shown which player had which track. It would have been great to have two copies of each CD we used. Please let us know if have suggestions on how the process can be improved.
The order the following day (when I could not attend) was completely different from that on Saturday. Despite this, their results were similar to those of the blind format.
As to the split votes, we really did not have time to hear out why some voters like one player of the other. The close results between the DCS and Meitner, for example, could easily have gone the other way with a different group of voters. However, in two sets of pairings - DCS/Opus 21 or DCS/Meridian - the results were virtually conclusive. The DCS is quite a player. The latest version of the Meitner performed well enough to earn the votes it received against the NWO. Along with the NWO 2.5T, this trio were in a performance class above the Opus 21 or the Meridian units.
I am lucky to have been a part of this San Diego event and I have no vested interest in any of the digital players evaluated. Thanks for the nice summary Pete. Unfortunately I could not attend the LA comparisons due to a prior commitment.
It is important to note the rationale behind the simultaneous voting via simple show of hands. This was done immediately after comparing any two players on a given track. We quickly, without discussions, listened to another track and evaluated the same two players. This approach guaranteed that no discussions took place until the comparison of each pairing was complete. All participants agreed that if discussions occurred prior to voting, comments from more outspoken, influential or respected individuals can potentially impact how others vote.
Both tracks used to evaluate the players are highly regarded reference recordings of acoustic performances and the recording process used minimal mic'ing and no compression. The first recording used had a mezzo soprano solist and choral ensemble with cello, oboe, flute, harp and organ accompaniment. The second track is a dynamic orchestral piece with lots of tempo, mood and dynamic contrasts using full orchestral ensemble as well as varied, individual instrument passages.
As to individual system preferences, we were split with about half using solid state gear while others had tube equipment. A few routinely connect their digital player direct to their amp and the rest use a preamp. This variance may account for the closeness in only one set of results, those between the Meitner and the DCS. However, in nearly all other pairings the results were fairly conclusive.
I'll try to answer some open questions and/or provide additional info...
Nsgarch - The AC regenerator (not only a passive filter/conditioner device) is the PS Audio Multiwave II+ and was set in Sine mode at 120V and 120Hz.
Ghostrider45 & Tedmbrady - Careful level matching was done for each pairing using a test disc with uncorrelated pink noise. Measurements were made with a highly sensitive, calibrated, pro spectrum analyzer with memory, averaging and EQ setting functions. [No EQ'ing was done and no equalizer was in the system. This is a feature on the analyzer that determines where FR adjustments should be made if one were to use an equalizer in their system]. Volume levels were matched in all pairings, except in one where a differential of no greater than 0.5dB was the best we could do.
Krisgel & Tedmbrady - For now let's just keep the focus on the digital players as our group worked really hard on keeping all other variables constant. The PCs, ICs and rack used, including the platforms on which the units stood, etc., were identical for each player. As to the system used, suffice it to say that it is full range and has been tuned, upgraded, improved and tweaked for the last 6-7 years and the endeavor to dial it in continues. This system is the same musical and well-resolved gear that has been consistently used by our local audio club in prior digital and other comparisons. At 20Hz and at 16.5kHz it is -3dB (-6dB at 20kHz) as measured at ear level from the LISTENING POSITION. Anywhere else in between 20Hz - 16.5kHz is virtually flat, with a +1.5dB measurement at 80Hz as the only thing worth mentioning. These measurements were done using the same uncorrelated pink noise track and spectrum analyzer mentioned above. [As an aside, we really should be demanding that reviewers provide similar FR measurements of their systems from their listening positions.]
Tedmbrady - We did not have access to the output impedance values for most of the players. So your question about the front end gear having an "issue (impedance, etc.) with the back end of the signal chain (preamp, amp, speakers)" remains open. However, level matching was strictly applied throughout all comparisons. Additionally, no performance issues with any unit occurred that would have made us look further into impedance or other incompatibilities.
Overall, it was a terrific, fun-filled 6-hour session. Thanks to the participants and especially to their significant others who tolerated our absence during this Saturday event. We had doubts about being able to get through all the comparisons across 5 players, using two different recordings, and doing so in a consistent, level matched and blind process. We not only accomplished this, but also had time for additional experimentation and listening. I find it very interesting that the results in a different system using completely different recordings mirrored those of the blind comparisons.
Interesting A/B results. Exactly how are they significant?
I guess the differences in high end products, especially SOTA products, are so gross that you can clearly hear all, or at least the most meaningful, of the differences between the the various contenders in a short period of time, ignoring such things as component synergy, individual preferences, ad infinitum. I wouldn't have thought that, but we have the result. APL is 'king of the hill'.
I will take Alex's advise and 'take this for what its worth'. After all it didn't cost me anything.
I would love to conduct a similar shootout, but with players that the average Audio nut can afford. Perhaps players such as the Resolution Opus 21, Audio Research CD3 or CD7, BAT Vk-D5SE, APL Modded Denon's, Ayre CX-7e, Cary 303/300, 306 SACD, Meridian, Etc. That would be equally cool.
I agree with Groberts. I find this sort of effort enormously useful and something only a large audio society can undertake. My only caveat would be the kit used. Presumably this was determined by what the members could bring along, but a really wide variation in prices and not suprisingly, the most expensive came out tops. A similar exercise with comparibly priced kit, perhaps modded compared against unmodded too, would be even more illuminating. Top marks for the results posted though.
Call me a skeptic or whatever you wish but the little hairs on the back of my neck tingle when someone starts bashing an objective shoot out. If it acts and talks like someone who reps the products that lost, then it probably is.
The last SACD test Pete is talking about was; the NWO-2.5T plugged directly to the wall (no power re-generator since it sounded much better that way) while the EMM Labs Signature was plugged to the power re-generator (since its performance deteriorated direct-to-wall and was much better using the PS Audio re-generator) and both units connected direct-to-amp. The DCC2 Signature has a built in preamp which also allows for direct-to-amp connection.
Pete did not have the chance to attend the shootout which took place the next day (Sunday) at specially made dedicated "Audio Bunker" (Dedicated audio facility designed by George Cardas, built with Audio Only in mind and acoustically treated by Reeves Audio) in Hollywood, LA where we compared again the dSC, the EMM Labs Signature and the NWO-2.5T. That shootout was not blind. The same PC, IC cables and power conditioner were used for the 3 machines. The audio system consisted of Nearfield Pipedreams speakers with 18 inch separately powered subs on each side and all-tube Lamm monoblocks (not sure the model) powering the Pipedreams towers. The system used an electronic crossover. The EMM Labs Signature owner brought his own power conditioner (I believe it was passive) which he found to work best with his EMM gear. We used this power conditioner for all players under test. The EMM Labs owner, the owner of the Audio Bunker and representative of the San Diego Audio Guild were the witnesses. The levels were precisely matched using PAA3 analyzer and Pink Noise, just the way it was done in San Diego. The results of this shootout were consistent with the San Diego shootout.
Finally, we hooked the NWO-2.5 directly to the wall (without the power conditioner) which, just like in San Diego, brought further improvement in sound quality.
Last but not least, I should mention the generous hospitality of the Audio Bunker owner in Hollywood! The 16 year old Lagavulin scotch was really a special treat too! Thank you!
Disclaimer:
This is NOT advertising but the actual truth! The witnesses of the Hollywood shootout would like to remain anonymous (at least that was my understanding) so this is the reason for posting this information. Please take it for whatever it worth to you!
Pete, thanks to you and your friends for orchestrating this shoot-out. This type of review is a needed supplement to the on-line and magazine reviews. Your summary was very well written and your process beyond reproach.
I have owned a NWO2.5 since late December and in my comparisons with the dcs single box and EMM Labs combo I came away with the same conclusions. The NWO2.5 gets me the closest to the live music experience than any digital player I have heard...and on DVD-A/SACD it's as good or better than many of the very good vinyl rigs I have heard. I hope you or some other group will do a shoot-out with the NWO2.5 and some SOTA vinyl rigs. Thanks again for a very well done job. The entire audiophile community benefits from this type of information.
Pete, I am intrigued by your observation that "I had the uncanny feeling that I was in the presence of real music." I have heard this from others about the top APL models and I'm trying to get a handle on what it means. You talk about air and spatial cues, but I wonder if you can elaborate even more about that uncanny feeling.
First of all, thanks to you and the other members of the San Diego, Los Angeles and Palm Springs audio communities that organized this shoot-out. Your review was extremely well written and it looks like your process was beyond reproach. These types of reviews are too rare in the world of high end audio and a greatly needed adjunct to "professional reviews".
I have had my NWO2.5 since late December and while I have not had the luxury of comparing it to all of the machines you tested (only the EMM Labs Comb and DCS single box units), I would agree completely with your conclusions, especially the last part that the NWO2.5 brings you the closest to real live music (particularly on SACD and DVD-A). I listen to live music on average once a week - acoustic and amplified - and the NWO2.5 brings me the closest to that experience of any digital source I have heard...and I would go a step further and say that when playing DVD-A and SACD on the NWO2.5 it meets and exceeds several of the SOTA phono set-ups I've heard, although I want to be clear in saying that I have not yet A/B'ed it against the best of the best vinyl rigs.
On that point, I hope some organziation like yours takes on the challenge of doing a similar blind shoot out of the NWO2.5T and a few of the SOTA phono rigs. I think the results would be very interesting indeed. Perhaps SOTA vinyl would still win, but I think the performance gap has been reduced considerably over the past year and will continue to shrink going forward.
Thanks again Pete for taking the time to do this shoot out and share the results.
Meitner or Meridian Signature? In listing the digital gear involved in the shootout you said Meridian 808 Signature but in the comparisons you say Meitner Signature. As stated by Tvad, this stuff is "cost no object" of reach for many of us (the blue book value of our cars can't touch some of these players-mine included and I drive an Infiiniti.) Maybe if I ever move into a different tax bracket......
You described the variables (the players) nicely, thanks. However, you didn't mention anything about the setup. What speakers were used? Preamp? Amp? How were the volume levels calibrated (even a db off can make the louder one seem considerably more robust, etc)? Do we know if any of the players have an issue (impedance, etc.) with the back end of the signal chain (preamp, amp, speakers)? Did everyone who voted have similar system tastes (i.e tube fans, ss fans, etc.). The reason I ask this last question is that (huge unfair generalization coming, simply for simplicity sake) the APL is known to be more yang than yin (organic, liquid, some A-goners call number "2" sound), the others are known for their speed and detail (number "1" if you will). Neither is necessarily or inherently better, just different. Assuming everyone in the sample came to the test with similar tastes, then the results are more accurate. Me, I'm a number "2" guy and would likely pick the APL in the first ten seconds, especially if the preamp and amp were quality ss sound. Thx Ted
In my posting, I mentioned that, after the blind shoot-out, we ran two additional listening test. The first was plug the APL directly into the wall, bypassing the power regenerator. The second was a head-to-head comparision of the Meitner and the APL playing SACD. I neglected to mention one very important additional variation. Between the two above tests, we dropped the pre-amp out of the configuration for the APL only and plugged the 2.5T directly into the power amp. This produced a MAJOR improvement in the performance of the 2.5, and was probably also a factor in what the group called the "SACD blowout".
What a GREAT write-up and a fabulous, state-of-the-art shootout! I like the King of The Hill method, followed by subsequent experimentation to substantiate your findings. Very Cool!
I would love to conduct a similar shootout, but with players that the average Audio nut can afford. Perhaps players such as the Resolution Opus 21, Audio Research CD3 or CD7, BAT Vk-D5SE, APL Modded Denon's, Ayre CX-7e, Cary 303/300, 306 SACD, Meridian, Etc. That would be equally cool.
You must have a verified phone number and physical address in order to post in the Audiogon Forums. Please return to Audiogon.com and complete this step. If you have any questions please contact Support.
