You don't need to spend a fortune. Use LPs sound better than most audiophile reissues. Re: Turntable and Phono stage...just buy used.
Personally I prefer turntables that use a detachable headshells. It makes changing, aligning and handling cartridges so much easier.
Get the best turntable and phono stage you can afford.