First, I agree totally that this is a bad idea. A year or so ago my nephew called and asked me the same question. He wanted to connect two amps to the same speaker terminals and only power up the amp he wanted to use at the time. I told him it was a bad idea and strongly advised against doing so. And I feel like this is an inherently bad idea. But aside from opening the door to possibly accidentally powering up both amps at the same time, I really don't know specifically if there are other likely problems.
Can someone with the tech chops chime in?
Bill