More power for moderate listening levels?

This is not an easy question to answer, and it would depend on particulars that are impossible to factor in, such as your own personal preferences and priorities.  My own priority is for the amp to sound good while playing at lower output levels where the vast majority of my listening takes place.  For me personally, I happen to favor the sound of low-powered triode tubes, and even in the solid state realm, the amps I like tend to be lower-powered, such as the gear from First Watt.  This means using fairly efficient speakers (mine are 99 db/w efficient).

Millercarbon raised a very point--lower efficiency speakers cannot really achieve good dynamics no matter how much power is fed because of thermal compression.  As more power is fed to the voice coil, it heats up and that heating increases resistance which reduces the amount of power that can be delivered to the driver.  The less efficient the speaker, the greater the compression.  My difference with his opinion is that I would not be so dismissive of the Harbeths just because they are not efficient; there are plenty of other reasons to like any particular speaker even if it cannot do "really loud."

It might be the case that the higher powered Pass amp will improve the sound, even at lower levels of play, but, I have my doubts about this because caused by some inherent superiority of having much more power in reserve.  This is something you will have to try for yourself to see if it is the case.  A long time ago, I auditioned an am rated to deliver something like 50 watts/channel.  The same company made a 200 watt/channel amp that the dealer said was very similar in design and parts quality, but used more output devices to achieve the higher power capability.  I actually preferred the sound of the lower powered amp in a direct comparison.

Can you be more specific when you say you hear strain or lack of clarity at higher volume levels — maybe give some specific examples?

I agree that auditioning is the only way to go to see if one amp is better sounding than the other. Just curious about the effect of power at lower volumes.

At high volume I think the amp is staining because horns and cymbals are becoming harsh and not as musical. Overall, music is just not as pleasant and I gather it's not just due to high volume irritating my ears.

Appreciate the thoughts y'all.

How loud are you listening?  If you're trying to do 110dB, then there are better loudspeakers than the Harbeths.  Also, you mention that you're further away from the loudspeaker and dancing (presumably standing), both of which will alter the tonal balance of the loudspeaker.  Do the Harbeths sound strained at your normal listening position at high volume levels?

What is the volume level setting when you crank it up and notice the strain?  The XA-25 (same amp used in the INT-25) was tested by Stereophile at 80Wpc into 8 ohms and 130Wpc into 4 ohms. It does switch to class A/B at higher power output levels, so perhaps you are hearing that change?
I'm using the Pass INT-25 with Sonus Faber Electa Amator III speakers, 87db, 4 ohm, probably a more difficult speaker to drive than your Harbeths.  With my chair at about 8 feet from the speakers I usually set the volume in the range of 40 to 50 depending on the CD (63 is the max volume). I've also wondered if a higher power amp might drive these speakers better, although I don't think I'd want to give up the INT-25 for moderate listening levels.