Are you using a 1/8-inch stereo mini-phono plug adapter? This plugs into the headphone jack and the Mac will recognise if you are using headphones or a digital interface.
You only use the Mac volume if using headphones. For digital output, the preamp in your system controls volume.
You only use the Mac volume if using headphones. For digital output, the preamp in your system controls volume.