Source vs Target: where is the weakness?

The source is the key. However, there are real world trade-offs. For example, speakers are harder to sell and ship so should you get great speakers first and hold onto them? Or, what if a really great amp deal came along? Should you then build the rest of the system around it? Since these kinds of situations occur all the time, the pragmatic answer is normally "it depends". That's what makes this hobby so much fun. All else equal, most would argue to start with the source and build it out as Kinsekd suggests (I say "most" based on past threads related to this question). I look forward to other responses.

P.S. It's critical to have an overall philosophy or approach in mind regardless of how you go. To Kotta's point, the system is the key. For example, do you want to go the SET/efficient speaker route? Are you interested in near field listening? Listen to as many alternatives as possible before embarking on the journey regardless of where you begin.