I had a cartridge with the same problem once. I didn't notice it at first and had mounted it and played it. When I observed it under magnification, I was like, wow. Not good. Surprisingly, the cartridge still sounded good, at least to me anyway. I adjusted the azimuth and this made the cartridge track properly with no problems. I used this cartridge for several years with no other problems. No damage was done to my records. Lesson learned though, I inspect every new cartridge thoroughly before I mount it to the head shell. I'm not saying to keep the cartridge. I would opt for a free replacement from wherever you bought it.
Dang my luck
I ordered a new Denon 103r. It came in yesterday and after I attached it to my headshell and gently removed the guard a noticed the cantilever was leaning visibly to one side. If you look at the pic looking down the cantilever you can see it's not bent but looks like it goes into the body crooked.
https://flic.kr/p/R3Ts5F
https://flic.kr/p/PX4FXh
https://flic.kr/p/R3Ts5F
https://flic.kr/p/PX4FXh