The lobing is actually due to having 2 spaced sources of the same signal at the same frequency. It isn't really caused by the crossover, but by the drivers. (Just being pedantic here.)
I don't see directivity patterns (lobing) as a huge problem - provided that the designer is aware of it and communicates with the user.
As long as you know how the speaker is to be used and the proper listening axis, it isn't really very necessary to worry about directivity. (As long as your listening room meets some minimum requirement, which I'll discuss later.)
The ultimate is omni-directional speakers, with Bose being a good example. They design for uniform frequency and power response throughout the room. In doing so, they utterly trash the signal. They can't accurately reproduce a transient.
Siegfried Linkwitz is also very concerned with frequency and power response. But he ignores time-related (phase) elements. His designs are highly engineered, but I doubt they are very accurate. Hmmm... I guess I would say they are good designs, because I respect all the work and thought he's put into them, but I just don't agree with the compromises he has chosen.
Mithat Konar has some interesting comments on the MTM designs. http://www.birotechnology.com/articles/VSTWLA.html
I don't agree with Mithat on a lot of things, but he also puts a lot of thought and effort into his work.
Would it be best to have flat frequency and power response throughout the room? Probably. But there seem to be more important things to take care of first. Pat McGinty said that once you ensure your design passes transients correctly, the frequency and power response fall right into line.
Stereo requires a fairly limited listening position. You have to be equidistant from both speakers. It starts getting into psycho-acoustics, and that's complex and not something I fully understand, but I know a few general things. You need to accurately reproduce the amplitude and time spectrum of the original signal. That requires phase-coherent speakers.
Phase-coherent isn't the best term. Time-aligned isn't either. Minimum-phase? That gets a bit closer I guess. Maybe I'll just stick with phase-coherent... it gets the idea across without having a connotation of a certain method like time-aligned does.
At any rate, accurate loudspeakers don't really need to worry too much about the rest of the room, just the listening position. If there are major problems with the room, then that needs to be solved separately. I you can't remove reflected sound in the room, then you need to get enough delay so you won't smear transients. (I've seen several papers that detail the amount of delay needed to ensure that a reflection is not perceived as part of the original signal.) You also need some amount of attenuation, from distance or damping material in the room.
If you live in the right area, you can take your stereo system outside and hear it without all the room reflections and resonances. This can be quite revealing.
Where there is disagreement is in whether you design to minimize room effects, or allow for them or even exploit them. A lot of time is spent on this. There is probably room for different philosophys here, because some people just may not have a good room to listen in, and designs that work with this may at least give them decent sound, if not accurate sound.
A typical ballroom is very reflective. You can certainly hear the echoes. But, the delay is great enough that you hear them as echoes, not as part of the original sound. (The distance also reduces them in amplitude.)
A small bare room has little delay between direct and reflected sound. It's probably the worst place to listen. Bose is probably correct in this type of room because you're not going to hear transients properly anyway, so you might as well try to have even frequency response and get some part of it right. :-)
Listening outdoors gives you long delay, plus significant attentuation, without being so anechoic that it's disturbing. (I feel that when you're in an anechoic chamber, you can't hear where you are or what type of environment you're in, and this is what is disturbing. It's very unnatural and not like the environments we evolved in or are used to.)
Most people can't listen outdoors, so they need to ensure their listening room is as large as practical, and also well-damped. (Damping or room treatment can be said to increase the apparent size of the room.)
Phase and transient problems seem to be on the "leading edge" of things, with room problems being on the "trailing edge." I feel that it's most important to have an accurate "leading edge," and then you can do whatever you can to improve the "trailing edge." I think Linkwitz explicitly says the trailing edge is most important.
Is phase audible? The studies I've seen haven't insured that there was any "control," in other words there wasn't phase-coherency to begin with. They just took a system with unknown phase properties, then added phase changes to it and asked people if they could hear a difference. There was one Master's thesis on this topic that I saw that was so flawed it's a wonder they let the guy get away with it. I hope to find a more scientific study at some point...
I have found that designs that attempt to insure phase-coherency sound more lifelike than ones that don't. So I'm convinced, even if others aren't. :-)