I prefer XLR vastly over RCA primarily because it offers the possibility of operating with balanced lines. The risk of interference is so greatly reduced that it's the standard in professional audio: microphones, mixing consoles, equalizers, signal processors, tape and disk recorders, etc., all have XLRs and have for decades. XLRs are also mechanically robust and have a locking mechanism.
RCAs are small, relatively inexpensive and convenient, but that's about all that I can say in their favor.
No connector should have a "sound." No connector should audibly affect the audio signal. If it does, it's defective.
I also like XLRs because you can buy great balanced cables from studio supply houses for a fraction of what typical unbalanced cables cost from hi-fi boutiques.