I sure hope that is not the case for most audiophiles, if not for all. The key is to refine your preferred audio sound as you keep moving up over the years.
I was fortunate to learn with, and patronize real audio shops in the late 70s, 80, 90s, and on, until the gradual decline of brick and mortar stores. I’m not sure how I’d have ever learned what my ears now know without those stores and guys. Now, good high end shows can help; and losing RMAF makes me cringe.
And if you still can’t find the sound you are looking for after a long time, your process for looking/listening may be off, or your preferences are a moving target (good luck if that’s the case-sounds like a revolving door inventory of gear).
Over the years I worked on speaker and component upgrades as funds permitted and finally truly found the sound/speakers (and electronics) I love and can happily keep in retirement with no real yearning to upgrade. Still, I can think of better versions of my system, usually moving up the speaker line one prefers, or smart electronic component upgrades. But I don’t worry at all now.
And, even with my systems of the late 90s and over the next 20 years, I’ve never been unhappy and ‘still looking’ at any time (save one or two gross system mismatches based on buying used without hearing or understanding the sound). I did move from ML electrostats to Dyne Sapphire 3-way dynamic speakers but got lucky and settled on smaller 2½-way used speakers that are much more refined. Happy and done but would take the next larger model if offered a pair…
The constant craving can end, honest.