Let's take it from the system user's viewpoint:
-an audiophile is one who needs music to listen to his system;
-music-lover is someone who needs the system to listen to the music.
In this light, "sounds good is all that matters" can apply to both categories.
I guess we're all music-lovers tuning our systems to better enjoy the music