The one thing I haven't seen mentioned is that most, if not all measurements are static. A sine wave (or two in the case of IM distortion) is fed into the device and deviations from the input are noted, or a power output recorded, etc.
However, music is complex and dynamic. There is no guarantee that a static measurement accurately represents the device's behavior when handling a complex signal. One of the best examples of this were high-feedback solid state amps from the 1970s that measured well but sounded shrill and harsh. Things are better these days, but there is still no perfect correlation between static measurements and musical performance.
Then, there is also the interaction between multiple components. An amp that works well with a speaker with a benign impedance curve may not sound near as good with a difficult load.
That said, measurements are still important, particularly when designing and manufacturing products. For manufacturing, measurements are the only practical way to maintain consistent quality control as there is no way to listen to every unit as it comes down the production line.
Finally, don't think I'm a pure subjectivist. We are humans, and subject to both conscious and unconscious bias that have little or nothing to do with sound but which still affect a person's perception. Just because the subject is hi-fi doesn't mean those issues don't exist.