Perhaps the most annoying myth in audio of 2025? Talking about Loudness!

Volume matching in A/B comparing is important. Usually, louder will sound "better," even by 1 dB. 

However, I'm of the opinion that one should choose volume levels to match what one thinks the source probably sounded like. Tool? 100 dB is about right. Mahler at 90 dB? Sure. For big Mahler finales, with a gong on stage for instance (e.g., in the Eighth), even more than 90 dB would not be too loud. 

But if you crank it up for small chamber orchestras, or string quartets, or solo piano, or acoustic guitar, etc., you misrepresent the source. To my ears, fine tuning the volume as I begin to listen is as important as fine tuning the balance (which is important for getting instrumental placement and soundstage right).

if A:B testing is not carefully volume matched the test is of dubious value. In doing a quick switch, louder will generally sound better.
for longer listening sessions, I think it’s a matter of personal preference and will depend on the type of music. 

"It is said far too often that the louder speaker will sound better." Huh? I've never heard that, or at least don't remember hearing that...maybe it wasn't said loud enough. I have horn speakers that can go very loud without strain (hifi and pro stuff) so speaker distortion isn't an issue around here also since efficient speakers require less grunt from the amp. I listen at various levels based on how I feel but really not particularly loud, but so what? To each their own. And that concludes my statement of the obvious.

The OP is confusing the use of volume in a/b demonstrations (where listeners will often prefer a subtly louder replay of a track) with listening levels in general.

I have never seen in print or heard an assertion that a majority of people prefer louder playback on a sustained basis.

On "loudness", there is quite a variation in the levels that different recordings are mastered at so even listening at a consistent average SPL can require quite different volume control settings.

@ghdprentice I never heard that one... It is definitely not true in my experience. 

Do you use a potentiometer volume control? Most people do.

Your amp runs full open and the voltage must be  REDUCED to normal listening levels. Any type of potentiometer 'applies the brakes' to this signal by restricting voltage and losing resolution in the process. So, as you turn up the volume you are releasing the brakes allowing more signal to flow and sounds better.

A pre-amp with lots of gain takes the 'reduced resolution signal' and then amplies it. You get loudness but with distortion added. Some people like the perception of added dynamics. I don't.

A potentiometer directly affects the voltage level of an analog signal by acting as a voltage divider, while a digital volume control modifies the amplitude of a digital signal without introducing any additional noise. So, newer, more expensive digital volume controllers seem to be prefered in higher end gear.

So, if you use a potentiometer the higher the knob the better.