I tried a HiFiBerry DAC Pro+ a while ago in a secondary system. I thought it was remarkable for the $50 price. However, the sound was not as good as through a couple of audiophile DACS (Auralic VEGA, Classe CP-800). By comparison, the HiFiBerry was rather dull sounding, less transparent. I didn't notice distortion (though I didn't use it very much), but I definitely experienced loss of detail, as you did.
It's not your imagination.
It's not your imagination.