One of the gradual degradation regimes most people don't think about is demagnetization of the driver magnets. I have thirty year old Altec Lansing Voice of the Theater Speakers that I sent the woofers to Great Plains Audio for rebuild. They reconed, remagnetized the drivers, replaced the voice coil spiders and speaker suspensions. They are as new.
The demagnetization occurs if you drive your speakers hard for long periods of time. The heat and vibration will disorient the magnet domains. This doesn't happen overnight, it's a gradual and accumulated process that can take years.