Ok, $5000 is the upper limit of your budget, you would prefer less. You want a Turntable and CD playback.
You would consider upgrades as money becomes available.
Lets start with CD playback. I would purchase a CD transport that plays into a DAC. I own a Cambridge CXC which retails at $600 however can be found discounted. There are others, check out Cd transports on this forum. Audio Advisor sells the Cambridge and the Audiolab, both aforementioned herein. Another option is to purchase a CD player having digital output. Either way you can take advantage of upgrading the DAC should you choose.
Regarding the TT. I would allocate ~$700 to $1000 for this source. You are purchasing 3 components, table, arm and cartridge. Most in this price range have compatible table and arm with a budget moving magnet (MM) cartridge. You will also need a phonostage between the TT and amplifier. There are some budget phonostages available. Project, NAD, Schiit to name three.
Next we consider the integrated amplifier. As you know they can be found as line level only, or having a internal DAC, and/or internal phonostage. Now this is where some discussion regarding possible future upgrades comes in. If one owns quality CD transport and turntable, one can upgrade the DAC. If one owns a quality TT, one can make improvements upgrading the cartridge and/or the phonostage. The key is to have an amplifier that mates well with the speaker. Speakers, room size, and loudness levels will dictate the power need of the amplifier. There are a number of speaker what would play well into your room when provided with a 50-100wpc integrated. If one were to find one having both an internal DAC and MM phonostage. That said it is easier to find one that has DAC only. This would save money on separate components and additional cables. This approach would be a good starting point when on a sub $5000 budget, allowing greater expenditure on sources and speakers, significant improvements at a later date and while allowing you to experience your music collection now. I will mentionone such amp, the Yamaha A-S701. I retails at $750 and can be found discounted. It provides 100wpc, and has internal DAC with coax and optical input and a MM phonostage. I own one and find it to be very good. I have not used the phonostage.
Ok, the choice of speakers is the most subjective of choices and therefore hardest for others to recommend. Again when on a budget I suggest those the reasonable sensitivity, 86-90dB and 6-8ohm impedance. There are so many that would work well within your situation. The Wharfedale Linton speaker has been mentioned.. There is a review in The Absolute Sound. Cost is $1200 w/out stands, $1500 with. An east to drive speaker with a 50-100wpc amp. Music Direct carries it and allows for returns. Speakers are one place to save purchasing used, or dealer demos.
Moving forward I advise you to check out local stores, Music direct and Audio Advisor as online outlets, and then consider starting specific threads on this forum concerning component options.
A CD transport playing into the Yamaha A-S701 using it's DAC driving a pair of Wharfdale Lintons on there stands would make for a very nice system at a retain cost of $2850. Leaving considerable funds for a vinyl system for a total under $5000. This to serve as an example, not the only option component wise.
Good luck in your endeavor and please keep us posted.
