Regarding issues with less than stellar recordings, reference rigs do to lower noise floors, detail retrieval, and extended frequency response are not tolerant of less than stellar recordings. You hear all of the engineering faults clearly that are masked on vintage systems.
I agree @dover that vinyl distortion at high playback levels is usually not flutter (speed inaccuracy) but rather acoustic feedback. I would look first at TT isolation and speaker placement relative to the TT. If it is flutter, it should be evident at low volume as well as high volume. This does not seem to be the case since you state “it sounds so good” … until you turn it up.
What does you reference rig do better than your vintage rig or vice versa?
I’ll be spending some time over the next few days with my vintage rig. The Cult Love album, which I bought on CD when it was released, sounded terrible on the main rig after I bought a vinyl copy a few weeks ago. So the album was demoted to the vintage rig discography because I love the album. Here the album has the bass I remember from decades ago. It sounds awesome!
The vintage rig plays less than stellar vinyl pressings better than the reference rig. I greatly enjoy listening to these albums on this rig!
However, I noticed that the vintage rig is far behind the reference rig regarding turntable quality. There could be other factors at play regarding the turntable.
I want keep turning up the volume on Love. It sounds so good that I want moreI However, I’ve reached that critical point where flutter can be detected if the combination of phono pre output level and receiver volume are too strong. I don’t have that problem with the reference rig.