I have used Rel subs with my MG3.7s with some success. I think one key to a successful pairing of any dynamic sub with Maggies is what Johnny b said above--Don't try to do too much! If you allow the subs to just provide a little foundation at the low end and don't try to achieve + 6 dB at 20 Hz (don't laugh, this is what a few people want) you can make it work with the REL R series subs. Yesterday I played a few minutes of music through the sub without the 3.7s playing. It is amazing how little in an absolute sense is playing, yet how much it adds subjectively. Getting a good integration is easier with two subs in a stereo arrangement than with one blended mono. Also, if you set up the Maggies to do all they can at low frequency (out 4-5 ft from the back wall for instance), it will make the task easier. CWs advice on room set up is excellent, though I respectfully disagree with his absolute avoidance of subs with Maggies.
Finally, I suspect (but haven't heard in my system) the Fathom subs may be superior to the RELs, but at the cost of the Fathoms, you would do better moving up to the 3.7Rs. While they aren't particularly strong below 40 Hz, they are much, much better overall than the 1.7s.
Finally, I suspect (but haven't heard in my system) the Fathom subs may be superior to the RELs, but at the cost of the Fathoms, you would do better moving up to the 3.7Rs. While they aren't particularly strong below 40 Hz, they are much, much better overall than the 1.7s.