What do you consider when setting crossover freqs?

After all I have read, I am still unclear as to whether the cross over for a sub is set at the frequency where the mains/fronts taper off (i.e. the non-flat portion of the frequency response curve) or it is set up where the spec say the mains/fronts are responsive to, say down to 60Hz). Or none of the above.. or other factors I do not know about.