Same amp brand, same board engineering, the bigger amp with the larger power supply and power reserves is going to sound better. That power is always there, at low or high volume. Push your left finger (speaker) with your right finger (50w amp). Then push it with your right hand's palm (400w amp). For one you almost always get a better damping factor and bass control. Think about it. and when the going get's tough it's even better.
But, different brands sound different, and quality varies. I recently moved from a 570w/1140w amp to a 160w/?s watt smaller power amp (although with large supply and capacitance). The Odyssey amp bested my old W4S (decent sounding to me, before...) amp in five very audible ways almost immediately. Stunning. Power had nothing to do with it. The bass got way better too. :-)
As for why buy low-efficiency speakers, I'll go with @Clearthinker's view on it (grin). Honestly, my ears have never heard a high-efficiency speaker that didn't seem 'in my face' to me, but that's just my ears... I'll just plod along with my 85dB Raidho D2s (room equalized flat to 30Hz and smile lots as they play.