Strange Turntable Question

I wonder if this is a common occurance or a problem I need to address. On my Gyrodec, when the motor is on and the platter spinning if I take the tonearm by the headshell handle and squeeze it very hard between my thumb and 1st finger I can feel vibration. Just holding it without pressure with one finger no vibration is present. I wonder how much this vibration would affect the sound when playing a record.

thanks