Finally a couple years ago I added a tubed amp. Audio Research finally put biasing under microprocessor control and lengthened the tube life to around 3,000 hour. Wow, why had I waited? The sound profoundly better. It I had to do it over, I would have made the switch decades ago. My reasons for not switching were the same as yours. In retrospect the reasons were smaller than I perceived. The final huge upgrade for me was adding a tubed DAC… End to end tubes can be very synergistic. I had a tubed CD player earlier which made CDs listenable but at a huge loss of detail.
I made many runs at digital… good audiophile quality CD… then DACs. I did this on my main system and my headphone system which frequently served as my research lab (tried dozens of components). Over the last thirty years the improvement has been continuous and over the last ten years there has been so much progress that digital can now be as satisfying as analog but at about 15% - %20 initial higher cost. This cost premium drops a little ever year. My current streaming vs vinyl ends are equally satisfying… the analog end has a tiny bit more detail… tiny. I have 2,000 vinyl albums. And virtually an infinite amount of music available via streaming. So honestly, although the initial cost is a bit higher… the nearly infinite available music at no cost favors streaming.
I have very good components… but the difference is similar across the high end cost spectrum given choosing compatible and complimentary components (the same exact challenge across the high end systems of any kind).
So, it I had to choose between getting rid of my analog end or digital… I would definitely loose the analog end. It has taken 35 years to get here.