Well,
With the vacuum off, the highs are more pronounced, more "splashy", the bass is a little loose. I think this is due to the record being free to resonate due to the stylus/groove interaction.
With the vacuum on, things seem to be duller, but during careful listening you can hear details that were obscured before. There is more detail in hall echo reverberations
Your reaction may depend on downstream components too, cheap speakers (not to mention a cheap low resolution cartridge) may very well obscure the differences.
The effects are subtle but I think they're worth the upgrade. Certainly better than spending thousands on wire.
One thing that I have noticed is that the thicker the record the effects of the clamping seem to diminish.
Audio magazine did a great report on the vacuum option that was very enlightening. The reviewer commented on how much the "needle talk" was reduced when the vacuum was on.
Cheers.