For me, it comes down to how much are you enjoying music / audio - if you enjoy experimenting with wires or gear, more power to you even if it's at the expense of reading books, going to the theater or whatever other optional pursuit you might have. There's nothing about one optional past-time that makes it inherently better or worse than any other. On the other hand, if it's actually causing you more anxiety than enjoyment, stepping back for a breather and letting it re-establish itself as important could be a good thing. Personally, I don't have the time to experiment with all the possible enhancements I'd like to, but someday I may. If I could afford it and keep my wife thinking I was sane, I'd find time to experiment with lots of different speakers, at least for a while, but I don't want to spend that much money for the experience.
At the end of the day, nobody should apologize for enjoying something, even if nobody else understands why, as long as it's legal, etc. -Kirk