I recall my friend Murray Zeligman who designed speakers telling me that it often wasn't frequency response that caused brightness but poor crossover design to the tweeter resulting in ringing in the crossover region.
Another possibility is flat response all the way out. Dynamic speakers should roll off slowly to compensate for changes in power response. As the dispersion narrows with increasing frequency the energy is concentrated resulting in bright response on flat recordings.
And, of course, there are poorly mastered recordings deliberately made bright to attract attention but that become tiring on well designed speakers.