Dereck, Don't worry too much about the amp. From the Stereophile review:
"Fig.8 shows that the RB 1080 clips quite suddenly. However, this is at very high power levels. Using the conventional 1% THD definition of clipping, the Rotel puts out 291W into 8 ohms with one channel driven (24.6dBW), 558W into 4 ohms (24dBW), and even 800W into 2 ohms (23dBW)! Considering its modest price, this is one powerhouse of an amplifier. But, as Rotel recommends in the handbook, the RB 1080 is best used with speakers whose impedances don't drop much below 2 ohms.John Atkinson"
Your amp has PLENTY of power for those speakers (especially in that lively room you have!). Earlier comments about the amp refer more to the overall refinement and not its ability to power you speakers. I truly believe that most of your problem lies in the room and speaker positioning. Best of luck and let us know how it goes. Dick