Is the color in the amp, the preamp, the source?
To add to my earlier post, and to address the OP more directly: I think there is a general consensus among audiophiles that colorations exist in every component in the system. Perhaps the greatest contributors to coloration are the two things you didn't mention - the speakers and the room. Regardless, some audiophiles conclude from the fact that every component is colored that neutrality does not exist.
I, for one, believe that neutrality exists, in the sense of degree of absence of coloration. Although every component is colored, not every component is equally colored. Some components are more colored than others. Components that are less colored are more neutral, in the sense of degree of absence of coloration. This issue was debated at great length in the thread I mentioned in my earlier post, so I will not belabor it here. I mention it only by way of addressing your question
Would it not be preferable to add any coloration (if one so chooses), warm, cool, what have you via the source and/or the preamp?
As I understand it, you are asking whether there are any generalizations about selecting components for neutrality or coloration based upon their function in the system. For example, is it better to have neutral speakers and a colored source, or a neutral source and colored speakers? Answering those kinds of questions would be helpful for achieving the kind of system synergy that several other posters have (rightly) emphasized. Unfortunately, my suspicion is that there few, or possibly no, generalizations about these questions that hold true across a wide variety of systems.
Having said that, I do believe that, when assembling a system, it is valuable for an audiophile to ask himself how each components is colored (i.e. deviates from neutrality). The answer will always be speculative to some extent, since we can listen to components only in the context of a whole system. But it is valuable nonetheless, because, even if we get it only approximately right, understanding how each component is colored will help us get closer to the sound we are looking for. This is particularly true when the colorations in question are not euphonic. As for the best place to deliberately add euphonic colorations to a system, I suspect that that is a question that can be answered only through experimentation, and even then it will be a matter of taste.