What's the cost of an entry level vinyl setup?

I'm just curious how much it would cost to get the equipment required to make vinyl worth listening to. I know players are available for cheap, but would anticipate the sound quality to be the same.

What would be your recommendation to go with a system worth about $5,000 in total? Is there a significant advantage to vinyl at this level? I've heard vinyl on high enough equipment to know that at it's best it's better than digital, but am not sure what it would cost for the average joe to get a quality sound.