No, you're not imagining things. At least, I don't think you are---but maybe we just inhale too much of the same stuff!!
Check out Issue #150 of Bound for Sound. The editor, Martin DeWulf, has long experimented with polarity and directionality of cables and wires. In this issue, he reports on his experiments w/the directionality of IC's and found that the best sound depended upon the manufacturer (no surprise) as well as where the ICs were placed, ie, between the source and preamp or preamp and amp. How they sounded really was brand dependent and location specific. Eg, the same IC may have sounded better in the direction of the arrows between the source and pre but would sound better oriented in opposite direction when placed between the pre and amp.
BTW: DeWulf was one of the first to discover that the relative polarity of the PC cables w/in the same system made a difference some 10-12 years ago.