Of course tubes are psychological. But amps sound different too. Different amps all have their own characteristics. One manufacturer's solid state amps sound different than another's. And so do dufferent tube manufacturers. But tubes in general have particular characteristics compared to solid state.
They have higher distortion unless the solid state has very low loop feedback which does occur but very, very rarely. And if the tube amp is single ended it has extra high distortion which is mainly second order, which fortunately is musical but still not accurate.
With very few exceptions in out world of low efficiency speakers(with exceptions) tube amps have less power than solid state(there have been a countable number of high power tube amps like Melos) which can limit your choice of speakers. And most flat response speakers are low to mid efficiency.
Tube amps have low damping factor and this interacts with speaker impedance to alter system frequency response. Check the Stereophile response curve with simulated load. Basically where the speaker impedance curve goes up with low damping you get a corresponding system frequency response affect.
And if your tube amp(or very rare solid state amp) is single ended it is inherently non-linear. If you put a sine wave through it the bottom half of the curve doesn't go as low as the top half goes up.
Of course solid state isn't perfect. It's probably more accurate but if solid state foibles bother you more than tube foibles then even though they may be technically bigger foibles go tube. You're the one that will be listening to your valuable music for hours and hours