+1 for Academy Records. I lived in NYC for over a decade and in vicinity most of my life except for the last five years. Academy Records on West 18th has the best classical collection in the city by far. If you are looking for new (not used) records, also be sure to check out Rough Trade. That was still in Brooklyn when I lived there but it seems they got popular enough to move it into Manhattan.
