Depends what you're looking for... vinyl? CDs? Genres?
I buy Jazz, Classic Rock, Current Americana, Singer-Songwriter, Folk, Newgrass CDs from:
importcds
discogs (some new but mostly used)
Ebay
Amazon Marketplace (last resort)
(All of the above also sell vinyl)
There are also specialty shops that cater to particular genres.
Personally, I've found shipping speeds to be highly unpredictable.