I commonly hear people say that they want their system to sound like the "real thing". But of course that is never going to happen since you do not know what the "real thing" originally sounded like. What you mean, I suspect, is that you want it to sound like your interpretation of what you think the "real thing" should sound like.
@crozbo It helps a lot to use recordings you made yourself, because you were there when it was recorded, huh! 😀
If you even get a chance to try equipment that supports AES48 (which is likely a bit more expensive) I'd be interested to hear if you still hear the same differences.
When the standard isn't supported, the shield of the cable becomes the signal return path, just as it is in an RCA cable. So its more susceptible to noise and the construction of the cable itself. Not only that but you lose immunity to ground loops. Cable immunity and immunity to ground loops are two things that the balanced line system is supposed to eliminate, along with reduced noise (Common Mode rejection of noise; CMRR).
So if the standard isn't supported, you're likely to hear cable differences. I think this is why the single-ended vs balanced thing is an on-going thing in high end audio since so little of high end audio equipment supports the standard.