If the switch is configured the way you describe then you are not missing anything. Are you trying to compare the streamers based on their performance with a specific dac or on their own merit? If the streamers are built with their own internal dacs you can compare by running each to a separate input on your preamp- or through auxiliary inputs on an integrated amp if it supports multiple inputs. As @calieng mentions above- it’s better to keep the signal path apples to apples for your comparison. If USB is your thing be consistent with both during your comparison.
Best way to A/B two different streamers
I just purchased a new dedicated streamer after streaming tidal from my computer. I'm using USB to stream from both. Has anyone ever tried using a USB A/B box to switch between streamers to evaluate the differences? I know it would possibly degrade the signal but I'm only looking to hear the differences. My other problem is I only have one high quality USB cable but could use 2 cheaper cables just to hear differences. My DAC and streamer are both in a rack where the cable is hard to reach.