Amplifier used w/ headphones and no speakers connected, can it be damaged?

Class D amps it is no problem.  Class A and class A/B it is almost always bad.

If you are using the headphone Jack then most integrated amps would not send a signal to the amp portion and there is no problem. Should be easy enough to check. If using the speaker taps then you are finer as others have indicated 

In my 50 years of audio sales, the advice has always been…no problem with solid state, and no good with tubes.  One instance where solid state can have an issue is with amps lacking a common ground.  When I was in college, I had a Quad 33/303, neither of which had a headphone jack, so I bought a quarter inch female in-line stereo connector and soldered it together with some speaker wire and banana plugs to accommodate my Sennheiser 414s. The 303 was designed as a pair of bridged amp modules and didn’t like the grounds tied together…luckily I caught it before it overheated too much!  But if there is a jack, that won’t happen. Another kind of amp that would do that is a class D like many NAD models, but they provide HP outputs at the preamp stage.

As others have stated, running a solid state integrated sans the speakers will cause no damage. Most integrated amps with a headphone port automatically disable the speaker output when a headphone is plugged in. If your HL5's stop out putting when you plug in your headphones, you'll be fine without them. You will also be safe when swapping phones if you have more than one pair.

You can easily test this by leaving the speakers hooked up and plugging in the headphones.  The speakers should cut out with the headphones plugged in.  If that's the case there's no issue at all.  

And if you aren't going to use your Harbeths anymore, send them to me. ;-)