"Another possibility is that my two-phase Kelly drive controller is doing a better job" - it had better be!
Try tweaking the third harmonic control whilst running the platter, you should be able to find a minimum in the cogging torque transmitted.
"I'm inclined to conclude that an AC motor needs a small amount of compliance in the belt"
But how much? How does one calculate the optimal value?
I have just taken possession of some new belts which are far , far less compliant than standard Mylar tape. I am combining them with those "crappy coggy Hurst motors". I think you will be interested in the result. It will hopefully be at RMAF in October.
Mark Kelly