if both amps had automatic speaker disconnect relays in them, and you were extremely careful, presumably a setup like this could work. but it's an accident waiting to happen. As mentioned.. when accidentally both amps are powered on at once.
If you wanted to install a manual high quality A/B speaker selector switch... while a bit of a kludge, I guess it'd work.
If you wanted to install a manual high quality A/B speaker selector switch... while a bit of a kludge, I guess it'd work.