Most people outside these "high-end" parts would not consider my stuff low cost. For example, I just got a nice deal on a pair of mint Sonus Faber’s from someone locally (compared to other popular options you might read as touted here), but even then still more than most people would ever shell out for a pair of speakers. It’s really not hard at all to get top notch sound for modest cost these days, even with a very modest budget and very high audiophile-level type expectations going in. It just requires doing your homework and a lot of patience perhaps.
If headphones are sufficient to meet one’s needs, you really have it made! Do the homework and find that ~$150 for ear monitors and dongle + an existing Smartphone or tablet with streaming can get you sound that is reference standard by any objective measure quite easily. That would be the absolute lowest cost setup I could identify that is capable of providing reference quality sound. Disclaimer: reference quality may not be to everyone’s liking. That is more a matter of personal preference. In that case, it’s even easier, you just buy whatever sounds good to you and that’s all that matters.