What should I do

Get another pair of smaller Harbeths and set up a very near field (speakers 2-3 ft in frt of your chair) listening arrangement for late night. With a bit of practice you will be able to set up and remove in a matter of minutes.

Unless you already listen extensively to headphones, the change from speakers to phones could be unsatisfactory.

While I'm not sure about the smaller Harbeth idea, Rhljazz makes an excellent point about headphones. It seems the people who like them are addicted, but if you're really not much of a headphone listener, don't assume that you're just going to buy a great set of phones and a great amp and problem solved. Many phones, including ones that are allegedly "open", can still sound very constricted if you're used to speakers. Not to belabor the point, but that's what I like so much about the Grados - I really feel like I'm not listening to headphones. Still, I use them far less than I do the speakers.

Thank you very much for the advice. So many are correct as to my loving speakers and possible lack of experience with headphones. Yes when I've tried them, I've found them annoying . I have not heard HD800 and now off my list. I really need to audition Grado's and amp before investing on something I will hate. I will continue to confine my listening with Harbeth's to daylight hours while others at work. Thank you again.

Just one guy's opinion...Buy a Burson Soloist headphone amp and a pair of HiFiman HE-500 or Audeze LCD 2 or 3 (as your budget allows) and prepare to really enjoy the music.

What sort of late night hours and dB levels are you talking about. Are there neighbors above, below and/or next to your listening room? Talk to your neighbors about what you're doing, even invite them over for a listen. It actually might be better for everyone if you did some sound transmission inhibiting reconstruction to your listening room. Different equipment might not be the optimal solution.