Speaker grills on or off?

.
When listening, grilles off. When not playing, grilles on.

I don't know if there really is a difference, but just in case the grilles arent absolutely acoustically transparent, I keep them off when listening seriously.