Well said CD318
At the end of the day, your ear is really the only reliable personal testing tool. I have designed many, many amps of every type and have never found electrical measurements of distortion, IM, etc, useful except as a first cut to make sure you are designing in the right direction and have not made a gross mistake. The completion of designs always involved multiple listening sessions with musicians and music lovers to zero in on the best options and tradeoffs. Very seldom did the final choices agree with the 'best' measured specs. The only exception to this rule was in the bass response area--best reproduction there was almost always in agreement with listening tests.
This is not actually surprising. Experiments in listening back in the 50's, for example, determined that a very small amount of high frequency hiss improves the perceived enjoyment of the audio experience. (Say hello to Vinyl!) Other experiments have noted that the huge frequency non-linearity of human hearing greatly outweighs spec measurements--and that on a person-by-person basis. I have an acquaintance in New Zealand who has a million dollar sound system that is absolutely terrible. Every component was bought on the basis of reviews and price, and it is almost non-listenable. He, nevertheless, is very happy with his system and loves to talk about the price of each element while playing the music.
One final observation: Live music should always be listened to (a lot) by designers. The trap is to go for esoteric things that can be tested, like the best design for hearing percussive wood blocks, and so on. As a violinist and orchestra player I was very familiar with the sound instruments and even different specific instruments played by the masters. Pinchas Zukerman played my violin once, and I knew exactly what his Strad sounded like. Thus, when I designed I sought to reproduce the 'true' live sound experience--which is definitely NOT the same as designing by spec.
Cheers!
At the end of the day, your ear is really the only reliable personal testing tool. I have designed many, many amps of every type and have never found electrical measurements of distortion, IM, etc, useful except as a first cut to make sure you are designing in the right direction and have not made a gross mistake. The completion of designs always involved multiple listening sessions with musicians and music lovers to zero in on the best options and tradeoffs. Very seldom did the final choices agree with the 'best' measured specs. The only exception to this rule was in the bass response area--best reproduction there was almost always in agreement with listening tests.
This is not actually surprising. Experiments in listening back in the 50's, for example, determined that a very small amount of high frequency hiss improves the perceived enjoyment of the audio experience. (Say hello to Vinyl!) Other experiments have noted that the huge frequency non-linearity of human hearing greatly outweighs spec measurements--and that on a person-by-person basis. I have an acquaintance in New Zealand who has a million dollar sound system that is absolutely terrible. Every component was bought on the basis of reviews and price, and it is almost non-listenable. He, nevertheless, is very happy with his system and loves to talk about the price of each element while playing the music.
One final observation: Live music should always be listened to (a lot) by designers. The trap is to go for esoteric things that can be tested, like the best design for hearing percussive wood blocks, and so on. As a violinist and orchestra player I was very familiar with the sound instruments and even different specific instruments played by the masters. Pinchas Zukerman played my violin once, and I knew exactly what his Strad sounded like. Thus, when I designed I sought to reproduce the 'true' live sound experience--which is definitely NOT the same as designing by spec.
Cheers!