Yes, the Rotel can handle 4 ohm speakers, but only one pair of speakers at a time. If using both A/B then all the speakers need to be 8 ohms.
The Rotel puts out 100 watts at 8 ohms. It's only going to be more (probably 150-200W) at 4 ohms. You're totally fine there.
I have 104/2 speakers as well. I've run them with a Dynaco ST-70 tube amp, which is only 35 watts, and it sounded wonderful. But you are correct in that they really come alive with more power. I currently use a Mark Levinson 27.5 power amp which puts out 200 watts into 4 ohms and it's way more than enough; preamp volume knob never really needs to get above 9 o'clock.
As for HK, I wouldn't shy away from that either. SQ is always great from that company in my experience. I don't think you can go wrong with either. But if it was me, I'd go Rotel. Just make sure they have the inputs you need if you're going full A/V with it all.