If they sound fine leave them alone! Buzzing is caused be a rubbing voice coil - not a speaker cone!
When to replace old drivers?
I have speakers with 15+ year-old Fostex drivers. They sound fine, but the paper is discolored and one whizzer slightly crumpled (child's prying finger years ago)... At what point (shy of obvious failure like buzzing) would putting in new Fostex drivers constitute an upgrade?