Right on. It sounds like you're looking for One Integrated to Bind Them All. At your price point here're a few suggestions:
1. Cambridge Audio Evo 150 (~$3300)
2. NAD Masters M10 V3 (~$3000)
3. NAD C389 BluOS (~2500)
Each of these offers both streaming AND a built in phono stage. However, that phono stage will only accept MM cartridges. Truthfully, if you really want to enjoy your vinyl and you have that $5K budget, I would suggest something like this:
- a NAD C700 V2 integrated ($2000)
- the Rega RP 3 with its installed cartridge ($1200)
- maybe the Aurorasound Vida Prima or this Musical Surroundings Nova 3 phono stage.