Watch the series of videos posted by Dannie Richie of GR Research on You Tube (actually, the videos are made by Danny and posted by Ron of New Record Day). Danny is a speaker designer who does a lot of cross-over work for other companies, and offers his own DIY speaker kits.
Anyway, on the videos Danny explains in great detail the questions involved in the arrival times from multiple drivers, how different x/o slopes are used to get the drivers in phase with each other (where their outputs combine, rather than cancel) over as wide a frequency range as possible. The off-axis response of each driver is a major consideration in all loudspeaker design. If the off-axis response of a speaker is significantly different from it’s on-axis response, the sidewall, ceiling, and floor reflections will sound like another, different sounding loudspeaker (one defined by the off-axis response) is playing along with the direct sound of the real one. Not good.
The idea that time aligned drivers and 1st-order crossovers automatically great a great loudspeaker, and that in and of itself is all that is required, is an over-simplification of a complex situation. Watch the videos; Danny explains it all in a way that technically-unschooled audiophiles/music lovers can easily understand