The obvious difference between the biginning and the end would seem to be the zenith, but you have checked that, so that seems less likely. In fact, you've checked everything.
g m c has a good point, in fact, evan if you have got the anti-skate dead-on, AT the place you set it, the anti-skate is really most reactive in the sense that misalignment would cause tracking error before zenith distortion that you could hear. In other words, practically speaking, ellipticals are much more forgiving with zenith error and VTA, and when they are off, you would tend to lose focus rather than a less "smooth" sound.
ellipticals, good ones, do tend to be reactive to tracking force, subjectively, and too light a tracking force would cause the sound to be, to me, just what you describe. Anti-skate, would cause just exactly the same effect. They are inter-related, , and in my experience, can be quite touchy when I supposedly am SO close to being dialed in. When you are light on your VTF, anti-skate is more critical.
I also find, that evan though I am really near-sighted and have no problem setting up a cartridge with repeated results, that I can almost always do better by ear (although I sometimes get carried away and have to start over). It seems to me you are all aligned with your gauges, so maybe you should try your ears. I would try VTF first, at the beginnig of the record, then fine tune with anti-skate.
If this does not work, get back, I may have another idea.
