Speakers First?

Top advice from Audiogoners as well as industry experts typically suggest to buy speakers first, and then build a system around them. While this can make sense to me, my pragmatic brain kicks in to try to solve questions such as......
- How can I possibly listen to all the speakers I want to and compare them against each other (the listening environments would be very different in different listening rooms - including my own eventually)?
- Assuming I find a pair, do I carry them with me for store to store to interview source equipment? (Cartoon like scenes appear in my head thinking about this)
- Does this mean that source and amplification equipment matter less?

I could go on but you get my drift. I currently have a pair of Gallo 3's that I intend to keep.......until of course I hear something I like better :)