I use an older niles 4 speaker switch control ( they make a 6 and more) It is not powered. Just passive switches, it has inputs for two amp/receivers so i can have two receivers going in and four sets of speakers coming out. i can send amp A to any or all of the four sets of speakers or the same with amp B. No electronic circuits are introduced to the signal. It adds nothing or takes away nothing.
just switches diverting the signal from one of two amps to one or all of four sets of speakers.
I am not at home so i cannot give you a model number and it seems the new ones only offer one input. i am sure with enough research it could be found.
The douk mentioned by jlamb is similar but powered. I would only use a passive switch.