I realize that Hi-Fi stores are getting harder to find... But maybe a friend, or a store, will let you bring in the music, and media you want to listen to for auditioning (it will influence your decisions). A serious audio store will have ideas about entry level, after all, they want to see you come back, and back... Audition speakers first they are the biggest percentage of the sound quality, I think, and can be the biggest initial expense. An integrated amp, a decent turntable (literally hundreds of quality NEW turntables for under 1K), then a CD Player, (most manufacturers source from the same suppliers, when they don't you will see it in the price jump). Everything does not have to be new, but use common sense, if you see "no returns accepted", keep scrolling.
Then when you get going, upgrade according to your desire. A good Headphone will give you an idea of what can be accomplished. Unless you have run out of ideas on how to spend money and you think Hi-Fi might be the answer, then just throw your credit card at it and bask in your new found glory.