No, it's a non-issue. I defy anyone to hear a difference with the lengths used in the average domestic room. There are more phase issues and time smear from a speaker crossover and even more if the speaker is ported. As far as unbalancing the image, room acoustics will make a farce of that.
If the longer cable is a gauge heavier then even the amp's damping factor is maintained. Keep the cable to the nearest speaker short. Consider the alternative. If there is, say, 8ft extra, what will you do? Coiling it up is worse than having different lengths. Think inductor tucked away amongst all the other mess of wires.
Rather pay attention to your room' acoustics which will do more, much more, than fussing over equal lengths.