Caveat: Listening to mainly Jazz at volumes less than 85db, normally.
This explans in large part your outcome and the fact that you listen in a modest size room. Quality of your amplifier has more bearing than the quantity of watts. Your scenario makes perfect sense to me.
As others have pointed out, the limited volume control usage range is a reflection of excess gain in an audio system and is not due to too much amplifier power. The higher the sensitivity of your speakers or amplifiers the less preamplifier gain is needed. If your digital source or phono cartridge is high voltage output, the less gain you require.
Charles