Yes, horns are very low in distortion because of the very small excursion required to produce a lot of sound. But, there is no way of getting around the effects of reflection/interference within the throat and flaring sides of the horn, as well as the effects of the horn itself being energized and resonating. It is not simply a matter of treating these things as "bad" and trying to minimize the effect; the "art" is in making these effects work to produce the kind of sound one wants. If it were simply a matter of reducing these effects, such as preventing the horn itself from resonating, proper design would be easier. But, I have heard various attempts to heavily dampen certain horns that were completely disappointing. These things have to be "tuned" -- meaning resonance and coloration have to be made to work in harmony with the sound one is trying to achieve, just trying to minimize resonance does not work.
Yes, I was speculating that, perhaps, some of the perceived dynamics is a product of peaky frequency response. That is not to say that horns have to be peaky to sound dynamic--all good horn systems sound more dynamic than most other types of systems. This was just my observation that some of the more musically balanced and least colored systems that I've heard, such as a well set up Edgarhorn system, were also on the dynamically tamer side of the horn family. The most dynamic and lively systems I've heard were actually not horn systems but systems with full-range drivers on open baffles, and these have WILDLY skewed and peaky frequency responses--hence, part of my reason for suspecting that tonal balance is involved.
I agree that the better horn systems are, unfortunately, huge in size. The best that I've heard generally had extremely long throats and very large openings. This allows the midrange horn to operate down to quite a low frequency (the bulk of the music will be produced by a single driver, which minimizes the impact of the crossover on the sound). I wish I had the room for such a system. The other "problem" with horn systems is the difficulty in blending any woofer with a horn midrange. Most woofer/loading schemes either don't quite sound like they are matching the clarity and "speed" of the midrange, or they don't actually go very deep if they do seem agile enough. I personally prefer more agile and less extension, but, there is some sacrifice involved.
But, even though most of the "best" I heard were huge in size, some smaller systems do remarkably well. Old examples include Western Electric 753 systems, some "newer" systems include the Tannoy Westminster (yes, it is "small" by horn standards."