Do you mean drifts to the right? (counter clockwise) when lifted off the lp?
Any arm, with anti-skate applied, will drift to the right, as it has outward force, say 1.5gm, while when not in a groove, playing, the inward force the anti-skate is compensating for is absent.
The texture of the lifting arm will influence how much, or none, and if you lower the lifting arm when outside the lp's edge, the outward pull will be readily apparent.
This is why anti-skate should be set to zero when setting tracking force, then apply anti-skate.