Speakers are the most important because they come first for me when building a system. I don't know the power requirements of the amp until the speakers are chosen. The source can be chosen first, but it has no limitation of working within a system like the speaker and amp combination. A particular source will work with any speaker/amp. A preamp may or may not be needed depending on the source or if an integrated amp is possible. Since all the choices can be based on subjective preference, it seems the answer comes down to other non-subjective parameters. The primary one that comes to mind is the wattage requirement of the speakers.
That is not to say I chose my speakers based on their wattage requirement. Room is a good answer too because not all speakers will work in all rooms.
That is not to say I chose my speakers based on their wattage requirement. Room is a good answer too because not all speakers will work in all rooms.