I can enjoy how it sounds and listen to music at the same time if I'm listening to my main system with all the right pieces in place. When there is a suboptimal component in the string, it is easy to become less engaged listening to the music.
When I get bored after a long stretch of swapping different components in/out just to see what changes for better or worse, it's another trigger to stop and revisit other hobbies for a while. Stepping away from it for a while helps to hit the reset button in the brain for me. I've found this to be necessary to appreciate the system more too. I've learned swapping components may not always be better, just different types of sound. A good friend convinced me a while back it's all about personal preference and your mood at the moment. Stop and revisit later if you need to.
When the desire to listen to different music kicks back in, I restart for another stretch to simply listen to lots of different music again, and let the components be.