Interesting questions. I was under the impression that it's a little bit higher than 40hz before we perceive directionality, but not by too much. I don't know the exact figure though, or even if it is an exact figure and not a function of the listener's aural acuity.
Keep in mind that subwoofers are not only used to reproduce frequencies below the main speakers. Sometimes they are used to augment frequencies the main speakers already produce, but at too low a volume. That's why some speaker manufacturers recommend that their speakers run full range even with a sub. So you may not want to have the sub take over completely at a specific frequency.
If you use a steep roll-off between the main speakers and the sub, I would expect it to remove some intermodulation distortion. But the non-overlap in the crossover region may make integration more difficult. In other words, you may more readily perceive the differences between the characteristics of the main speakers and the sub around the crossover region where there is no overlap. You may also get a bit of a suckout or a notch effect at the crossover point. So maybe it improves one thing but creates another problem in its place.
I don't know. These are just speculations. I think your idea about the listening tests is best.
Keep in mind that subwoofers are not only used to reproduce frequencies below the main speakers. Sometimes they are used to augment frequencies the main speakers already produce, but at too low a volume. That's why some speaker manufacturers recommend that their speakers run full range even with a sub. So you may not want to have the sub take over completely at a specific frequency.
If you use a steep roll-off between the main speakers and the sub, I would expect it to remove some intermodulation distortion. But the non-overlap in the crossover region may make integration more difficult. In other words, you may more readily perceive the differences between the characteristics of the main speakers and the sub around the crossover region where there is no overlap. You may also get a bit of a suckout or a notch effect at the crossover point. So maybe it improves one thing but creates another problem in its place.
I don't know. These are just speculations. I think your idea about the listening tests is best.