As others have mentioned, woofers with foam surrounds may eventually need replacement of the surround. The life of such surround is usually shortened by exposure to sunlight and air pollution (particularly ozone). Other kinds of drivers should last longer. These days, tweeters are not built with magnetic fluid in the voicecoil gap; that fluid eventually dries up.
I’ve heard many systems that use terrific sounding woofers that have never had to have the surround replaced because they do not employ rubber foam (e.g. pleated paper surround); some of these drivers are more than 60 years old. The midrange compression drivers in my system are around 73 years old.