I don't see any problems in this case, except that my instinct would be to avoid changing the position of the amp's input select switch while the amp is powered up.
I'm sure Kijanki is speaking knowledgeably and accurately about the Benchmark. With respect to the amp input, as can be seen in the schematic (page 16 of this
large pdf download), the input select switch entirely removes the unselected input from the signal path, so what you are proposing should work fine. In some other designs the RCA input is connected directly together with the "hot" signal on XLR pin 2, and the switch shorts the "cold" signal on XLR pin 3 to ground when the RCA input is selected. If the design had been like that I would not recommend doing what you are proposing, for several different reasons.
Regards,
-- Al