I would have been in the camp of being sceptical about switches - especially the idea of putting a switch into a setup that doesn't have multiple devices. However, having assessed the effect of inserting a quality switch in two separate installations, I have changed my mind.
That said, looking at the two installations, the positive effect of an audiophile switch was greater where the setup involved more convoluted wiring and another switch upstream of the music server.
But even in the more purist setup involving a CAT 6 connection direct from the router, the switch still made a worthwhile improvement.