XLR/RCA adapters problem - Electrocompaniet

I have and Electrocompaniet EMC-1UP and a VAC Avatar Super.
I read that the EMC was designed to use the XLR output and sounds better when used this way. So, I decided to get the Cardas adapters to try it out since the VAC is single ended only. I plug them in and there's alot of noise which is reduced if I touch the cables which suggests a grounding problem.(?) Worse still is when I play music. There is alot of distortion. So I took a set of cables and reterminated them with XLRs (the hot and the ground) which was a bit better but still noisy unless I touched the cables. Then I plugged in the extra rca plugs while still connected through the XLRs and it cut the noise considerably. So the question is why go through all this trouble? Well the darn EMC does sound better throught the higher ouput XLR jacks. Any idea what's going on here before I open up the CDP and check if there's something wrong with the ground? Thing is is that I have good cables I would like to use but there's no way I'm going to butcher them. The EMC is still great sounding single ended but I would like to experiment.
I am relatively handy and can figure out simple elctrical problems but this is beyond me.
Anyone have any suggestions? Any other Electrocompaniet users have a similar problem?

Thanks