Which came first?
First for me had to be the wonderment of the realization of a basic stereo image. There are two speakers yet the voice is coming out of the space between them. I had listened and came to love the music long l before I became aware all of the other elements presented by “Stereo”.
I now realize that there is a reason the engineer is listed in the credits of an album. All those nuisances that I’ve learned to listen for are added by someone with a huge board with a lot of volume controls to add just the right amount of left, right volume and ambiance around the element. The sax you hear four steps left and six steps behind was never there. He was actually in a little booth somewhere playing into either a microphone or a group of microphones and then placed at a specific location after the fact.
Unless live recordings are recorded using a single binaural microphone (or two monorail microphones place in a fake head) and the performers aren’t individually miked the sound you are hearing is manufactured. So as I become more aware of the tricks used to trick my brain into building a facsimile of being there or bringing them into my room I pick up more and more nuanced things like reverberation and delay.
So the factors of picking up a listening skill depends upon the factors you are listening for and the desire to care about it. I use my wife as an example a lot because while she listens to music constantly the engineering aspect of it is of no importance more than "I hear the bass" but she definitely enjoys music as much anyone.