Will the amp or speaker get damaged?

Bryston makes a well-respected switch box for those purposes. It has three outputs for three sets of speakers (a single input, or course). I recall reading of a reviewer using one of these units to compare speakers claiming that it did not introduce an audible coloration to their ears.

Marco

The Niles Audio offerings include one with some kind of autoformer impedence protection, as well as units without like the Bryston. The latter requires amplifiers that are compatible with a 4 ohm load. Niles units are indeed much cheaper than the Bryston.

Marco