I also have a Headstreamer (picture below), which is a USB headphone amp.
With the Headstreamer, the noise is there but it is directly proportional to the windows volume settings. When volume is at 100%, the noise is the same as on the Yulong. But to listen comfortably, the volume is at 3%, and there the noise is very very small (almost none really). This is funny, I read that the recommended settings is to have windows sounds at 100% and reduce the volume in Foobar. If I do that: big noise.
Why is the Yulong producing maximum noise, as it the volume was at 100% even when not connected to a source and with analogue volume set to -60db?