Quite a few good places in Greenwich Village:
Rebel Records (319 Bleecker)
Generation Records (210 Thompson)
Bleeker Street Records (188 W 4th)
Record Runner (5 Jones)
Village Music (197 Bleecker)
Disk-O-Rama (44 W. 8th) mostly CD's
Rebel Records was a mess but the guy there will find it for you if you ask for something specific.
Record Runner mostly records.
Most stores don't have things labeled very well, mostly by alphabet.
I love NYC!