In my experience, damping is a two-edged sword. Too much can reduce bass impact and subjectively suck the life out of the sound in the midrange (and no, I don't have a technical explanation as to why the latter happens).
Imo you want to use no more damping material than is necessary. And in some cases, that ends up being zero damping material. Yup, I've had designs that sounded best in the midrange without any damping material at all, particularly if the internal dimensions do a good job of staggering and/or minimizing internal standing waves.