Some of the sounds speakers make, such as a sharp whack on a drum, a firm pluck of a bass string, a loud voice or voices, the plink of a piano key (transients), can require lots of watts for a very short period to reproduce the leading edge of those sounds optimally.
A 50 watt amp may or may not be able to deliver the necessary power to reproduce them optimally, depending on the speaker and your listening tastes. If you are happy with the sound you’re getting from your system, you don’t need a more powerful amp.
Also there are a lot of speakers that are power hogs that don’t sound good unless driven by a powerful amp.