Yes, the digital output from a streamer can vary widely. In this case, you are limited to SPDIF, either coax or optical. As an SPDIF server, the streamer supplies the clocking for the stream which can have a huge effect on the sound, good and bad. Assuming the DAC is better in the Mac, I would recommend using the coax out instead of the optical, and invest in a decent digital cable.
In regard to volume control, set the CXN to line level/100% and only use the volume on the Mac. This will give you the best results. This assumes you have a remote for the Mac of course.