Maybe 'wear out' is the wrong word. I don't think speakers are like jeans. Jeans wear out from use. I think the fatigue life of most of the parts of a speaker are going to be vanishingly high. You could probably play most speakers 24/7 at moderate volume or a decade or more. This is not true if you play them at super high volumes.
How long does it take a speaker to degrade is perhaps more accurate. Environmental considerations like pollutants and ozone in the home might cause plastics and organic materials to degrade. I suppose it depends on the environment. Oil filled parts may dry out.