A more pertinent question (to me, anyway) is: are audiophiles music lovers first, or second? I don’t listen to my system, I listen to music through my system. Just a means to an end, not the end itself. That’s why I don’t call "it" a hobby. Musicians don’t call acquiring and using superior instruments a hobby. They are tools, just like a hi-fi system, a means of access to either the making of or listening to music.
An easy way to answer the above question is to compare the amount of money you have invested in each, your music library vs. your system. One reason most musicians don't have good systems is that the little disposable money they do have goes into the acquisition of more and/or better instruments. They will actually go without food if need be. No music, no life.