Is Recording quality the real culprit?

OP   So why are we wasting our time spending so much money on audio equipment?

Because we want the best what is available for now or near future. We always want the best clean sound recordings and many audio companies have improved the sound. So far, this is the cleanest sound recording and audio playback system yet. Alex/WTA 

 https://wavetouchaudiopro.com/

@megabyte I suspect that jazz drew the better engineers and gear due to 'cred'; a more 'established' popular genre with the cognoscenti while early R&R was a bastard child of 'the kids' who just wanted loud & raucous...

Then, acts like the G. Dead appeared that not only wanted to have feckless concert sound with the Wall of Sound, but recording that reflected the same.....

Art pushed tech, which pushed art, and the cycle still continues.... ;)

I like to peruse the sound gear at live concerts and events to see what's being thrown at us.  Given the amount of screens and sliders that even smaller venues are cropping up, AI-run sound in the studios isn't far off....

"...soon you'll be dancing to "The White Zone is for loading or unloading Only..."

F. Zappa, "Joe's Garage" (I forget if it's Act One or Two....)

...or the sound of camera drones fluttering about....

....or the prices of tix to concerts.....makes going to a baseball or B'ball game cheap in contrast.....

"Living in the modern world is like having bees in your head....."  Firesign Theatre.

As a classical listener I find this not a big problem. 😆

Regarding rock v jazz historical recording quality, a key difference is that rock music is played on amplified instruments and is in general played a lot louder. This creates major issues with spill and achieving adequate levels of separation. In addition, you often have more instruments - or more instrumental tracks where overdubbing is involved - and more instruments sharing the same frequency range. All of this requires more outboard equipment and more complex mixing desks, so in terms of purity, the signal path is more compromised. And despite the fact that classic outboard gear from the fifties and sixties is now lusted after, a lot of it doesn't sound that great if you apply hi fi technical standards to it. Lastly, when multitrack recording was limited to four or eight tracks it necessitated signficant amounts of bouncing to get the desired track count, and bouncing seriously compromises fidelity. So it's not that surprising that historic jazz on average sounds a lot better than historic rock.

A BAD recording played on a good system still sound bad.