This is the difference between an Audiophile and a music lover. Music first for me. Now I can't take most bootlegs shows that used to be at so many record shows. Anyone who refuses to listen to a concert because it's subpar quality is more in love with his rig then the music it's supposed to reproduce.
Does it have to sound good for you to like it?
I listen mainly to classical music. The SQ of classical recordings is all over the place, not nearly as consistent other types of music. Recording large orchestras is a complicated and difficult endeavor. Smaller ensembles are easier to record. So, if you listen to a great performance of an orchestral (or any) recording but have trouble with the sound will you avoid listening to it?