For example, I currently use 4 different pairs of stereo speakers, several amps, sources (essentially multiple rigs), a multichannel rig, etc in 2 rooms. It covers the different flavors, signatures, etc with different types of music that I enjoy (while my moods fluctuate).
1 in 1 out when it comes to equipment/rigs is not a good idea at all (as you're noticing)....
Apparently, "it is a common problem that many struggle with", as indicated by one commenter. So, let us probe a bit deeper into the psychological make up of many typical dudes. Stop telling yourself that the one sht you got is the greatest sht on earth somehow (because you paid for it and you are somehow the most intelligent dude that ever lived n all). Admit to yourself/acknowledge the strengths and weaknesses of the sht you own (weaknesses exist in all kinds of cost no object gear too). Don't be a feral fan boy of one brand, etc i.e don't put all your eggs in 1 basket. Keep an open mind and keep some options open.