Hi Drew,
I think that your question is put in an exceptionally perceptive manner.
One major factor that I think is relevant would be the Haas Effect:
http://en.wikipedia.org/wiki/Haas_EffectBasically, our hearing mechanisms "latch on" to the leading edge of transient waveforms, and give them disproportionate emphasis relative to what may follow a few milliseconds or a few ten's of milliseconds later. We evolved that capability to aid localization of the source of sounds that may arrive at our ears via both a direct path and (slightly later) via reflections.
The Ayre's no-feedback design approach would, everything else being equal, reproduce the leading edges of transients more cleanly than an amplifier that uses significant amounts of feedback. That is because of the non-zero amount of time it takes for the signal to propagate around the feedback loop. During that amount of time the input may have changed significantly, and the correction being applied by the feedback process may be based on out-of-date information, so to speak. What is called transient intermodulation distortion is one of the adverse effects of that.
Creating an amplifier that uses little or no feedback while still performing well in other respects (linearity, low output impedance, low harmonic distortion, etc.) requires both very high quality parts, and a lot of care and intelligence in the design.
Also, for any given amount of harmonic distortion that the amplifier may produce, its structure is very significant. It's well recognized that low order even harmonics (such as 2nd and 4th) are euphonic and musical, while odd harmonics (3rd, 5th, etc.), and especially high order odd harmonics (9th, 11th, etc.) are the opposite.
Harmonic distortion is primarily the result of non-linearity in the transfer function of the amplifier, linearity meaning that a plot of output amplitude vs. input amplitude should be a straight line, within the amplifier's power range.
I'm sure others will contribute lots of other thoughts on your question, because there are obviously many factors that are involved, but in my case these are the ones that come to mind first.
Regards,
-- Al