I'm not sure what you consider a really good set of headphones or a difficult set to drive, but I have Beyerdynamic dt880 cans, which supposedly fit both criteria. I also have an Anthem AVM 30 and can use its headphone jack to drive these cans with more than enough volume (without having to turn the Anthem's volume way up, like I do when using my CDP's headphone jack) - and the sound is extremely good IMHO. So, don't buy a separate headphone amp until you first try the D2. Enjoy!