Hooking up 2 amps to one set of speakers

I have a HT setup and I would like to use a 2 channel amp for 2 channel audio use using the same set of speakers (B&W 804's). The obvious choice of adding a 2 channel is is to get one with a HT Bypass, but I am finding many other amps I like without this feature. As a workaround to a 2 channel amp without the HT Bypass, I was thinking of hooking up the HT receiver to the speakers using Banana connectors and the 2 channel amp to the same set of speakers using Spade connectors. As long as there is only 1 signal going to the speakers, I see no problem doing this. Could impedance be an issue? Any other concerns with doing this?
thanks to all.