Phase Distortion - Speaker/Amplifiers

I'm trying to get a better understanding of the impact that phase distortion has on the final sound quality and what is really going on in simple terms.

When I think of the phase relating to speakers my understanding isn't much beyond swapping the speaker wires around such that the left and right channels would be firing opposite of each other.

Bombaywalla tried to explain it in another thread, but I can't say that I understood it.