This hobby/interest doesn't have to be expensive. At least not in the thousands if your listening habits don't demand it. I don't need or want concert levels in my space.
Lots of options at reasonable prices and compactness too.
I recommend a Keces E40 integrated, suitable for alot of good stuff online with a usb connection (CD quality). A nice phono stage and two RCA inputs. Remote included.
Pick up some good budget speakers, new or used (could get more upscale speakers), can add a dac, turntable, etc. Don't need much more unless you go bonkers in this hobby and start going down the rabbit hole. And that alone doesn't guarantee better SQ or pleasure.
BTW, it's a small powerhorse that weighs 10 lbs.