I always listen critically because I want to know what the system is doing well, what could use improvement, and maybe most importantly, what the engineer is trying to convey (and maybe not convey?).
I want to hear all those small changes that would go unnoticed to most.
I want to hear the engineer "touch" the pan pot.
When a singer doubles their voice, barely audibly, in the background, I want to be able to hear that for what it is, and where it is.
On overdubs, I want to hear the "punch in" when it occurs.
I want to hear the buzz of the amps from the performance, though I could do without the snare rattle.
I want to hear when the noise gate opens and closes.
I want to hear the "easter eggs" many of which will never be picked up through casual listening on a lesser system.
The list goes on, but these are the things that I find enjoyable and keep the music fresh and most are only picked up through critical listening.
BTW, if it matters, I am a retired consultant/scientist who specialized in noise and air quality studies for CEQA compliance.