In Ann Arbor;
ENCORE RECORDS (on Liberty St.); HUGE and virtually overflowing stacks of vinyl. Priced within reason, but they are aware of what stuff is worth. Good used CD selection too.
WAZOO RECORDS; this is around the corner from Encore. Not as large, but a nice little shop.
P.J.'S RECORDS; another good one. Good jazz selection.
for new vinyl, School Kids Records used to have a lot of vinyl for sale, though I think they have phased out much of it. It's five or six doors down from Encore, so it's definitely worth stopping into.
I have yet to pillage Detroit for vinyl, but I know someone who used to work at a place called Car City Records that you'll definitely want to check out.
Good luck and enjoy!