Here are the human hearing/perceptual rules that are most important regarding this topic. There are three. They are:
1) the human ear/brain system used higher ordered harmonics to determine sound pressure
2) Nearly all forms of distortion (harmonic, IM and inharmonic) are translated by the ear/brain system into tonality
3) The ear/brain system has a complex series of tipping points; one of them being that it can easily favor tonality caused by distortion over actual frequency response variation.
The implications are immense. Now the other important thing to understand is that most of what we know about human hearing physiology has occurred in the last 40-50 years.
Our test and measurement regime OTOH is based on hearing knowledge dating from about 1960 and ignores most of the research of the last 60 years.
So finally we can discuss distortion in amps. It should be obvious now that distortion is far more audible than the older textbooks might have you understand. The lower ordered harmonics contribute to 'richness', 'warmth', 'bloom' and similar audiophile comments. The higher ordered harmonics (in trace amounts due to the 1st rule above) contribute to 'brightness', 'hardness', 'brittleness' and similar audiophile comments.
Usually loop negative feedback is used to reduce distortion in amplifiers and also has the secondary aspect of controlling output impedance. The primary reason for using it is to guarantee flat frequency response on a variable speaker load (the effects of rule two are ignored). This concept was developed by MacIntosh and EV in the last 1950s and early 1960s. By 1975 this idea had dominated the industry. I call it the Voltage Paradigm as the amplifier has to behave as a voltage source in order for this to work. The prior art is known as the Power Paradigm and is still in limited use today (SETs, horns, planar speakers and generally tube amps with little or no feedback are examples).
The problem is feedback is known to not only suppress lower ordered harmonics but the price paid is higher orders are added, in many cases clear out to the 81st (ref.: Norman Crowhurst). In addition, intermodulations can occur at the feedback node, generating IM and inharmonic distortions (IM and inharmonic distortion are sometimes the same thing, sometimes not- both contribute to brightness).
Amplifiers that are linear enough to operate without loop feedback often do not contain much in the way of higher ordered harmonics. SETs (up to about 20% of full power) are a good example, although they do seem to represent a special circumstance worthy of their own thread). If feedback is applied, the higher orders will be present even at lower power levels (and in an SET the distortion is often unmeasurable at lower power levels).
The second harmonic often attributed to tubes is not something inherent in tubes (transistors actually make more). It is really more a function of the topology of the circuit. You can avoid the second harmonic by building an entirely differential/push/pull circuit and this is how most transistor amps avoid it. This can be done with tubes as well.
The ear does not care too much about the lower orders (2nd 3rd and 4th) but it seems to care a lot about the higher orders (5th and above). Its been known since the 1930s that the 7th contributes to a metallic quality in the sound, even in small amounts. So you can have a fair amount of distortion (5% at full power) and if its all lower ordered harmonics the amp might sound just fine. OTOH 0.01% of only higher orders is going to come off more likely as a bright amplifier. This is why two amps on the bench can have the same bandwidth but one might sound bright and other not.
IM distortion, unlike THD can be objectionable even in small amounts. This is also true of inharmonic distortion (an example of that might be intermodulations associated with a scan frequency, which might occur in a digital system, a SMPS or a class D application, although the latter two seem to be fairly sorted out now as opposed to 20 years ago!).
Now if its not obvious by now, since the understanding of how the ear/brain system works is pretty well ignored by the audio industry, the result is that the spec sheets often wind up being a good example of the Emperor's New Clothes. IOW they make the amp look good on paper but lacking a correlation in sonic performance.
It is the contention of the Power Paradigm proponents that the tonalities generated by distortion often play a bigger and more objectionable role than do actual frequency response errors, which can be managed by careful amplifier/loudspeaker matching.
The tube/transistor debate seems to revolve around the aspects of this conversation. The bottom line is that there really isn't a simple answer and for that reason audition is often the best solution. If you find that you have a preference (for example, many people like tubes, which is why this so-called 'obsolete' technology is still around half a century after being declared so; obviously the market wants them for some reason and so is keeping the tube industry in the black) then you will also find that certain speakers favor certain amplifiers and thus equipment matching is simply an issue that can't be ignored. There is no one amplifier, tube, solid state or Class D, that can play well with all speakers. For more information see:
http://www.atma-sphere.com/Resources/Paradigms_in_Amplifier_Design.php
In the end, you alone will have to sort out how to approach the issue of creating a system that is at once musical and neutral.