The low power amp with say 50 watts cannot harm the speaker able to handle 500 watts, unless maybe if you drive the 50 watt amp so hard that it is clipping a lot. Then if clipping is bad enough the output looks almost like a square wave. This is kind of like, imagine you are doing bench presses. Only instead of the sine wave full lift you do a square wave and try and hold it at half way. How long till your arms are shaking? That's pretty much what happens with speakers. Power pushes the cone out part way, holds it there, distortion increases, a lot of it is very high frequency, and he combination of square waves and extra high frequencies burns out your tweeter. This is why almost always the tweeter goes in these situations.
Now if you really do use too much power, even really clean power, that is another problem. Because dynamic drivers are a voice coil inside a magnet gap, with a spider and a surround designed to hold the coil so it moves like a piston perfectly straight and on-axis. Which works fine within a certain range. Too much power and the excursion is too great, the spider or surround can flex the wrong way, the coil goes out of alignment, and it doesn't take much the gap is real small, voice coil gets bent and you start hearing a scraping sound, or maybe knock or rattle, depending on what happened.
Your third failure mode is heat. Power heats the voice coils, and if they get too hot the insulation can burn, leading to a short. Or the coil can deform, leading to scraping, rattling, etc.
These are the real reasons for giving power ratings. But it's like megapixels with cameras, the last thing that matters is the first thing they feed you. Because too much trouble to explain so you understand what is really going on.
Now if you really do use too much power, even really clean power, that is another problem. Because dynamic drivers are a voice coil inside a magnet gap, with a spider and a surround designed to hold the coil so it moves like a piston perfectly straight and on-axis. Which works fine within a certain range. Too much power and the excursion is too great, the spider or surround can flex the wrong way, the coil goes out of alignment, and it doesn't take much the gap is real small, voice coil gets bent and you start hearing a scraping sound, or maybe knock or rattle, depending on what happened.
Your third failure mode is heat. Power heats the voice coils, and if they get too hot the insulation can burn, leading to a short. Or the coil can deform, leading to scraping, rattling, etc.
These are the real reasons for giving power ratings. But it's like megapixels with cameras, the last thing that matters is the first thing they feed you. Because too much trouble to explain so you understand what is really going on.