REL T/5i compatibility with Yamaha integrated?

A few weeks ago I added a new REL T/5i sub to round out the LF of my beginner system (Yamaha A-S501 integrated, BluSound  Node 2i streamer, Pioneer SP-FS52 speakers, TEAC TN-280BT TT, Pro-Ject Tube Box S2 phone preamp). I've followed the high-level interconnect instructions for attaching the REL Speakon cable bare red & yellow wires to the main speaker outs on the Yamaha but just can't get the REL to respond - I'm a newbie, but I at least know you shouldn't have to put your ear directly onto the sub to hear if it's working. The REL support folks have suggested using the Differential connection method outlined in the manual  (connect only the red and yellow wires but allow the black (ground) wire to float free, unconnected. No dice with that approach, either.

I do get sound when I connect standard RCA cables to the Yamaha's rec out and into both of the REL's Low Level and Low.1/LFE inputs, but then I don't have volume control through the Yamaha, and it just sounds overly muddy. I've also tried connecting via the Sub Out on the Yamaha to the Low .1/LFEI connection on the REL and that does work, but again the bass doesn't feel tight and I have to set the output level (on the REL) to about 2 o'clock to really hear that it's adding anything to the mains.

I've got the REL positioned on a carpeted floor at about 10:00 o'clock about 12" behind the left Pioneer floorstander.

As the Yamaha is a Class A/B integrated, I'm wondering why it's just not pairing well with the REL's A/B power amp...?  Would really love to keep it and figure out how to properly connect via the high level inputs, but now I'm second guessing the purchase - ditch the REL for something better suited to the Yamaha? Upgrade the Yamaha...?

Thanks,