Most of what you say is true and makes sense, I believe. But I've never had any problem with turning focus on rhythm (or anything other component of music) on and off - depending on how the recording is done, I can concentrate on each of these components, as well as how one component goes with one or more other components of the music - endless variety of ways to explore, especially if the music is very layered.
Can left brain- right brain balance affect what you hear?
We all know that some people hear a stereo audio image depicting a sound stage wide and deep with specific location of instruments and singers on one extreme and some hear only left - right - center on the other extreme. The rest of us are somewhere in between. A lot of this is probably due to expectation bias, but one other explanation occurs to me. I think it is partly due to left-right brain balance or dominance. The stereo recording process produces an illusion of the sound stage and is a creation of your mind. The right brain (creative side) is known to be involved with the melody of music and the left brain (analytical side) is known to be involved with the rhythm of music. In my experience creative people i.e., artists and musicians, etc. are more right brain dominant and more inclined to hear the detailed image. Technical people, i.e., engineers and builders, etc. are more left brain dominant and are less inclined to hear the image details. It occurred to me that if you focus on suppressing left brain activity the right brain may become more dominant. By focusing on suppressing your tendency to follow the rhythm of music the image seems to "bloom" and become more obvious. The difference is subtle as most things in this hobby are. I tried it and I think it works. Try it yourself and see if it works for you. After you have experienced this, it will be much easier to do going forward. I'm able to turn the rhythm focus on and off at will and hear either rhythm or melody or both as I please.
On the other hand maybe I shouldn't have had that last drink😊.