I ended up getting a Schiit Vidar and Saga S and A/B'ed them for a few weeks. The Schiit pairing was slightly less revealing than the Cambridge, but the sound was much more full bodied and a little darker/sweeter. In the end I decided that the Schiit worked better in my system. As many have said, I think it's important to consider the overall quality of the sound and the way different components work together. Though I could impress my friends with the detailed sound of the CXA81, in the end I didn't find it to be as enjoyable a sound as that which I got with the Schiit Vidar and Saga.
I later got also got a Bluesound Node 2i to stream music from, instead of streaming from my iMac with a USB connection to the DAC. That was a noticable improvement and I wish I had made that change before I returned the CXA81. I suppose it's possible that some of the harshness and forwardness that I was hearing was a result of jitter from the iMac USB setup. I would have been good to at least just learn from that experiment. Regardless, I'm really happy with the sound from the Schiit Vidar / Saga S streaming via the Node 2i, going to my Dynaudio Audience 52s. Pretty underwhelmed with the BluOS app, though, but that's another story.