You know, switch seating positions with your wife. :D
Seriously though, I’ve had untreated rooms create a kind of distortion effect above a certain listening level. If you two are sitting in the exact same place while evaluating it may be an acoustic phenomenon.
Otherwise, as recommended above. Try swapping speaker cables to test if it’s an upstream problem.
Oddly enough, if your drivers are rubbing thew would show up on an impedance plot. Dayton DATS V3 might show you exactly where the problem is. that may be too geeky for you, but I find it indispensable. A driver rubbing will show up as an irregularity in the impedance chart, though it may only happen at a level too high for it to pick up.