As others have described, the effective mass and the compliance of the cartridge's suspensiondetermines the frequency at which the arm will resonate. But, as Viridian mentioned, there are other, much higher frequency ringing that really affects the sound more. This is most effectively mitigated by having the arm as stiff as possible and by having the arm damp such vibrations and/or transmit that energy away from the cartridge to the base to which the arm is attached. Increasing stiffness and damping means more material in the arm and therefore more mass. But, too much mass is undesirable, and therefore, there is an inherent tradeoff between optimizing stiffness and damping and keeping effective mass within acceptable limits. Why is reasonably low effective mass important?
As you noted, you could easily balance a one-ton arm with a one-ton counterweight such that there is only one gram of tracking force applied to the record. If the cartridge/arm system were a static thing, there would be no issue. But, an arm must move to account for the up and down motion of the surface of the record (minor warps) and for the groove spiraling inward on the record (and side to side movement for slightly off-center spindle holes, etc). That theoretical two-ton arm may be statically pressing on the record by only one gram, but, in motion, it will have very high inertial mass (same as "effective mass"). It will take a lot of force to get that arm moving, and once it is moving in one direction, it will tend to stay in motion unless a large force is applied in the opposite direction. With a phono cartridge/arm, it is desirable for the effective mass to be low enough so that the cartridge/arm can easily moved by tiny forces.
As you noted, you could easily balance a one-ton arm with a one-ton counterweight such that there is only one gram of tracking force applied to the record. If the cartridge/arm system were a static thing, there would be no issue. But, an arm must move to account for the up and down motion of the surface of the record (minor warps) and for the groove spiraling inward on the record (and side to side movement for slightly off-center spindle holes, etc). That theoretical two-ton arm may be statically pressing on the record by only one gram, but, in motion, it will have very high inertial mass (same as "effective mass"). It will take a lot of force to get that arm moving, and once it is moving in one direction, it will tend to stay in motion unless a large force is applied in the opposite direction. With a phono cartridge/arm, it is desirable for the effective mass to be low enough so that the cartridge/arm can easily moved by tiny forces.