Downside of sensitivity?

I'm new to world of audio, and still puzzling out a number of things. One question in particular that I have concerns speaker senstivity. It seems as though higher sensitivity is a positive, i.e. it allows one to use lower-power amplifiers, sound is more relaxed, generally more flexible with equipment. Yet there seem to be a large number of quality speakers with lowish sensitivity. Given that speaker design seems to be an exercise in compromise (short of the cost-is-no-object level), what qualities does one give up with higher-sensitivity (say, > 91db)? Conversely, what qualities does the speaker designer gain with a less sensitive design(86-87 db)? I've quickly learned that there are no absolutes when it comes to audio equipment, but any thoughts as to the trade-offs in general? Tks.