Would there be any sonic downsides to this arrangement?I suspect that there probably would be, and I would not recommend doing this. The result would undoubtedly be significant distortion of the digital signal waveform, as a result of signal reflections that would occur due to the resulting impedance mismatch, and the impedance discontinuity at the end of the cable that is connected to the output of the unpowered source.
It may still function, and I don’t think there would be much if any risk of damage, but chances are the resulting waveform distortion would significantly increase timing jitter at the point of D/A conversion.
Regards,
-- Al