Geoff,
+1 your post
I have not had nearly as much upgraditus as others. I believe it is because I have chosen carefully and wisely. I don't try to achieve a sound or sugar coated flavour filled sound. I seek accuracy and total neutrality. My thrills come from the music and the recordings and realistic convincing presentation that conveys the true dynamics of live music and real instruments (requires more than 120 db of clean headroom). Audio systems should be like a great camera lens - as clear as possible.
If you stick to this goal rather than chasing different flavours (as you would with good wine) then swapping stuff is not necessary because the goal remains fixed and achievable in incremental fashion. Others will just say I have tin ears.
I believe most equipment swapping audiophiles are just chasing their tails because they DONT know what they are trying to achieve and end up chasing every pretty skirt they see, getting nowhere other than having a lot of fun in an aimless chase with endless equipment rotation.