When are they ever in a position to judge what the music they are making sounds like to others who are not part of their groupThe same could be said of non-musicians listening to music. Their preferences are unquestionably valid, but they are not going to have experienced listening to the music as a nuanced interplay in the same way that a musician does. First-hand experience changes one's perspective, so in the end the question becomes are you listening for fulfillment upon the design, musicality, or some combination of the two?
I would bet that no musician going into speaker building does so without scientific training as part of the process. So we're really only speaking here in theoretical absolutes.