Can you build a transparent XLR selector box?

No. Just get a line stage with sufficient inputs.
Every additional switch, jack, wire and cable usually adds extra grunge.
Interesting you mention patch bays. Make like one and plug/unplug

I know that the answer is yes, as in every professional recording studio employs patch bays.

So if every professional recording studio does it, then it must be transparent. Interesting.

So if every professional recording studio does it, then it must be transparent. Interesting.

Tongue in cheek

I know that the answer is yes, as in every professional recording studio employs patch bays.

But have you personally tried one in your hifi signal path and were able to notice any effect on your sound?

Its important to understand that most studio equipment supports the balanced line standard also known as AES48. Most high end audio home equipment does not- although there are a few exceptions which are rare. Because if this you'll find that a patch bay or XLR switchbox may well affect the sound.

You could check out Mapletree Audio (.com) for custom signal router solutions. I have a line router and a speaker wire router from them. Dead quiet, hand built, No signal degradation that I can tell. They will do single ended orXLR on the line router. Their prices are on the website.

This looks OK 3 in 1 out only $54 free post and this Douk mob makes OK stuff
https://www.ebay.com.au/itm/272978213010

Cheers George

I only ever heard one that didn't sound terrible, and that was the Manley skipjack.

RCA only though. 

I had the rca version of the Mapletree LR-1 line router custom built for me and it was excellent.  They also custom build the LR-1 Pro line router which is the XLR version.  

https://www.mapletreeaudio.com/