Would the same thing be the cause here as well?Yes, your at best halving the load impedance seen by the dac they would come back to the same dac buffer/s only depending how it's configured one could have the load of the rca and 1/2 of the xlr on it, the other the other half of the xlr, so it's load sharing could be bit lopsided also.
Best to use all from the RCA output or all from the XLR output and hope that either will have enough low impedance drive to handle the parallel load impedance of what your driving into.
Cheers George