Your question is surprising, given how many times I’ve seen you on the forum. You express a common misconception about critical listening. It’s not about habitually listening in a way that is always dissecting every single instrument or constantly evaluating every sonic aspect. That would indeed be exhausting and detract from enjoyment.
Critical listening is best understood as "listening with a specific, often temporary, goal in mind for system improvement." Think of it like this:
Imagine your audio system is a garage. You don't reorganize it all the time; you do it when you need to. At the beginning; when you've added a lot more stuff to it; when you've changed things around in it or have different priorities. You don't reorganize a garage most of the time. You just use it for other purposes.
Same with an audio system: sometimes, you need to "critically listen" to set it up or re-organize it. You might focus on a particular section (e.g., the bass response), identify what’s out of place (e.g., boomy bass), and make adjustments (e.g., move the subwoofer). This is a focused task with an end goal: a better-organized garage.
Once your "garage" (your audio system) is optimized to your satisfaction – perhaps the bass is now tight and well-defined, or the vocals are clear and present – you don’t need to keep "re-organizing" it. You simply enjoy the improved space.
Who does critical listening? Well, anyone seeking to improve their audio experience. This could be audiophiles, audio engineers, or even casual listeners who notice something isn’t quite right and want to fix it.
Does it bring enjoyment? Some enjoy the process of analysis. Others -- it sounds like you -- find thinking analytically to be irritating or difficult and not pleasurable at all. To each their own. But in either case the *outcome* is a more satisfying listening experience. Once you’ve addressed the issues, you simply relax and enjoy the music without your brain nagging you about imperfections. It’s about getting to a point where the "quality of what you’re listening to" sounds good enough so you can stop being critical and just appreciate the music.