I've never really understood the "critical listening" angle. That's basically the opposite of simply listening for pleasure. If you're listening critically, you'll hear things that sound good to you, but you'll also hear things that don't which will lead you to find different equipment. If you only hear positive things, your system is perfect for you and you'll never need to "upgrade" and we all know that's a rarity among this crowd.
"audiophiles listen to their equipment"
That quote is misattributed to Alan Parsons, as I understand. Anyway saying there's a problem with wanting good sound is like saying an instrument designer, aiming for beautiful sound, could not possibly be a music lover and is only interested in the sound of the instruments. I.e. the sound is inseparable from the music. For me the beauty of the sound, good microdynamics, and so on, are the "doors" to the meaning of the music.