You are lucky. Your anthem has two inputs.
So get a dedicated 2 channel preamp and hook it to the balanced inputs. When listening to 2 channel sound your yamaha isn't involved.
For AV, run pre out from your Yamaha front channels to the Anthem RCA inputs so all you ahve to do is switch inputs on the anthem.
You will have to figure out volume on the Anthem. You'll want to set it pretty high, perhaps 1200 or choose something easy to return to, and then adjust the volume balance for the speakers like you do with the Yamaha driving all channels.
Jerry