There are two quite clearly divided camps here.
Besides, that's when true audiophile rig shows real class - when it achieves an impossible task of making poor recordings sound tolerable.
Does it have to sound good for you to like it?
I listen mainly to classical music. The SQ of classical recordings is all over the place, not nearly as consistent other types of music. Recording large orchestras is a complicated and difficult endeavor. Smaller ensembles are easier to record. So, if you listen to a great performance of an orchestral (or any) recording but have trouble with the sound will you avoid listening to it?