I would be curious to know what actual volume you are listening at? A crude app for your phone will tell you peaks at your listening position, and if you are feeling inquisitive, you can measure at 1m from the speakers.
1 watt at 1m will give you 86db from your speakers. It’s math. As was stated above, every 3dB step above that, will require a double amount of watts.
The other thing, is higher powered amps, have bigger power supplies. They will have more power in reserve at lower watt output.
Provided the transistors, and the topology of the amp is such that it is operating in a linear manner at low output, they will sound “better” at low volumes than a lower powered amp with an inadequate power supply.
Which brings us full circle back to Nelson Pass, and the relevance of the first watt.