I definitely get it. When I got my first good table I tried a top rated $200 Phonostage. It sounded terrible. So at each iteration I doubled the cost. It wasn’t until $2,500+ I started getting much of the sound quality my system was capable of. Finally at $8,000 my phono stage was no longer “in the way”.
Then I upgraded my turntable a few years ago and I had to double again to get the most out of my turntable.
Being so close to the source it is really important. But it is always a matching game to get the most out of your system.