For what it might be worth and given that our systems are VERY different, here is my experience. I have a 75W tube amp (ARC Ref75se using 8 ohm tap) driving speakers with a nominal impedance of 8 ohms and a sensitivity of 91db-listening room (and 89db anechoic chamber). I have never seen the meters on the amp ever go higher than 5-10W. I usually listen with the volume setting on my preamp (0-103) (ARC Ref6) between 3 and 26, background and dedicated listening when I want to blown away so that nothing else exists. I have very "transparent" (NOT the brand) cables. Anything louder borders on painful and I am afraid of blowing something out. There is no where in this system to adjust the gain. I wish there was because I'd like to use less attenuation of the volume.
With respect to volume, lots more in the signal path matters. I do not stream. Source: caching server with its own internal SSD, Macbook pro running JRiver Media Center, hi-res files on computer or external SSD, and embarrassed to reveal you tube. I run only NOS (no over sampling, all processing is disabled in JRiver except volume leveling and PEQ - parametric equalizer). Volumes for each source are different. All sources run without attenuation.
Conclusion, to echo what many others have suggested: this needs a systems approach. The best solution to OP's original question would be found in the synergy of the individual components of the system as a whole. And for ultimate endgame satisfaction the solution must include the often forgotten details that matter like the room's dynamics and the listener's hearing, sonic & music genre preferences. It has taken a long time to sort through all this for myself.
SPL which answered all my questions (and is fun to play with) Toptes TS-501B from Amazon for less than $30.
Happy listening dear friends, Leon AKA Timbrehead