It really depends on how neutral and transparent each piece in question is. In a top notch system, the weakest link will stick out like a sore thumb...not so much on a system with multiple veiling and smearing issues. Plus, as mentioned, the amp/speaker interface matters.
At some level everything matters.