The quality of the cable is of first priority in each case where I have compared. I think it unlikely budget XLR will unlikely ever best high quality RCA. But the difference is strongly component dependent.
I currently have all Audio Research Reference gear. Audio Research recommends XLR. I have tried both… there was no obvious difference. Since I was upgrading, I went ahead and swapped my RCA out for XLR. But, I did it mostly just to be sure. Less expensive RCA cables made a big difference… in a bad way. I found mostly it was about the cable quality.