I have never heard a speaker that did not sound better with the grills off. However, some speakers look "but ugly" with the grills off.
Grilles or No Grilles
I've always found that 'no grilles' leads to a more accurate listening experience. I've had many different speakers using many different grille cloth and components and always, no grilles works better for me. Often many of my friends find that a WAF means grilles on. What do you think?