I suggest it is a piece by piece analysis. You just have to plug and unplug and listen. On some pieces I have heard absolutely no difference, whereas with others I do, albeit it may just be a matter of gain rather than quality.
Generally speaking, I would say you aren't doing any harm to the sound using one set of truly balanced XLR and one set of SE cables from a preamp. The only time I've heard a difference was with a couple of source components.