Which is the most important part of a stereo system?


My system consists of a pair of B&W 630's, an old Denon 50 watt reciever (DRA-550) from the mid 80's, a Marantz CD5004 cd player, and now a Pro-ject Debut Carbon DC Turntable.  I'm pleased with the speakers and the cd player and while the Denon sounds good it has some issues and I want to upgrade.

I'm planning on returning the Pro-ject TT and getting a higher end TT.  I'm also looking into getting a new amp pre amp with a internal DAC.

Is the source the most important? The speakers? 

Please Help!
klimt
What's wrong with you Denon and your Turntable?
Have you heard another system that you liked?
Personally, I think a stereo system is really just a decoding mechanism. The more information you can retrieve from your source the more information you will hear. 
Do you live in an apartment, condo or home? The reason I ask is, unless you have neighbors that love your music you'll find listening with headphones doesn't break leases. 
With speakers, then its a matter of the kind of music you like, how loud  you'll play the music and how much bass you want. Some amps and speakers do work better together than others, ie - low powered tube amps with efficient horn speakers, high power amps with less efficient speakers like Magnapans. Also with speakers, consider how big is your room and how close will you be to the speakers. But mainly it's the type of music you like that determines which components work for you. Having the most reveling system doesn't guarantee nirvana. Try playing 70's commercial rock records on a reveling system and see what it sounds like. 
Simple answer to your question - an integrated amp like the Outlaw Audio RR-2160 sounds great and has a DAC, internet radio and a FM tuner. With your turntable, try using a felt mat and check out the videos on You Tube regarding the removal of the shipping screws near the motor. (I did on my ProJect and it cut down on the motor noise and the felt mat reduced the static clicks. Also, clean your records and try plugging all your components directly into the AC outlets (replacing your ac outlets with hospital grade outlets helps increase the apparent bass). Have fun and enjoy the ride.
For many years I believed it to be the speakers, but after experiencing a hi-end system in a room before it was (properly) treated acoustically, then after the room was properly treated, the difference was nothing short of stunning.
Again, the system did not change, only the room. I think anyone who is serious about hifi and wants the best from the space that their system resides in should correct the space before trying to correct the system to their ears liking.
For me it's my ears.  Can't do jack without them.
But so far as the hardware chain goes source components first as numerous have said
@tweak1  Terry9, and anyone else who thinks cables are least important:
you couldn’t be more wrong"

So you think that cables are more important than the speakers? Amplifiers? Preamp? Phono? Turntable? Tonearm? Or cartridge?

And what is your evidence? I mean a real test, not a gut feeling.

I did the test. I built a special cabinet to be a Faraday cage, then used theoretically optimal 99.99% pure silver conductors inside teflon tubular insulators which barely contacted the conductors, which were separated by 50mm for near-zero capacitance. ETI connectors. Compared to premium microphone cable with ETI connectors. Difference negligible.

The specific model of epitaxial output transistors, from the same manufacturer, same technology, and same family, made far more difference. As did the resistance of the amp's circuit breaker. Lots of things make a difference, and even cabling does to some extent. But nothing compared to an entire component. And next to nothing in terms of benefit related to cost.