What should I do

Headphones aside, I totally understand your predicament of apartment living and maybe I can offer a suggestion. If you listen to rock this may be out, but I've spent a lot of time collecting music I can listen to late at night at low volumes - solo classical guitar, solo piano, violin/guitar, string quartet, etc. There's a lot of good stuff out there without excessive bass and drums that you might be able to get away with. I personally find it preferable to headphones, even good ones.

I am an ex-Harbeth owner myself and I would recommend you take a listen to the Grado GS-1000i phones. I basically consider myself a headphone hater and I much prefer listening to speakers in a real space. To me, whatever frequency-based or alleged sonic superiorities of other phones, IMO the Grados sound more like listening to my speakers and less like headphones. At the latest NY show, I listened to many different phones and, except for the Stax, I still preferred the Grados for that reason. Plus,they will sound absolutely wonderful with a Grado battery headphone amp for $350, which is a definite plus. You can always upgrade the headphone amp later if you feel the need.

While I'm not sure about the smaller Harbeth idea, Rhljazz makes an excellent point about headphones. It seems the people who like them are addicted, but if you're really not much of a headphone listener, don't assume that you're just going to buy a great set of phones and a great amp and problem solved. Many phones, including ones that are allegedly "open", can still sound very constricted if you're used to speakers. Not to belabor the point, but that's what I like so much about the Grados - I really feel like I'm not listening to headphones. Still, I use them far less than I do the speakers.

That will undoubtedly be a killer combo, but just keep one thing in mind. The Grados are really, really open, meaning you can hear every noise in the room, almost as if you were wearing nothing. You'll have to listen to them as if you were listening to the speakers - in a quiet environment. If that works for you, I would vote you proceed. Best of luck.