Distortion is interpreted as loudness so less distortion means louder will still sound good. You need a good recording without distortion as well as a large dynamic range to want to crank it and still find it engaging and enjoyable.
Volume Increase of decrease as system improves
Almost all of us have been on an upgrade journey with our systems. My question is as your system has gotten better has the volume at which you listen gone up or down. My personal experience is that it has gone up. I think because there is less distortion hence less fatigue at higher volumes.