Is speaker design art or science?

If you ask one audiophile what makes the perfect speaker he will say, perfect response, on/off axis, perfect distortion, perfect cumulative spectral decay etc. If you ask another, he will say its a matter of preference.

Depending on who you ask, its considered an art or a science. The problem is, it cant really be both. It can only be one or the other. 

Either speaker designers know what theyre doing.They know what theyre trying to achieve and how to achieve it. Or they dont. And they are just making it up as they go along. They are simply putting a bunch of parts together and seeing what the outcome is. 

Whats the perfect speaker cabinet? Is it one thats perfectly rigid? one that is well damped? or one that sounds nice?
Whats the perfect crossover? Is it one that measures flat on axis? Or one that sounds good?

Is speaker design a hoax?