Upgrading 5.1 home theater system

I have a similar sized room to yours and is also open on two sides to other rooms. I’ve got an SVS SB2000 sub and it’s fully capable of shaking the room when called for, and I honestly don’t feel the need for anything more. The sub is sealed and isn’t too big — it’s basically 14” square on the front — but it still gets down to 19Hz (-3dB) and is right at your budget (you can save an additional $50 if you buy from their outlet, and there are two unblemished subs available there now). It also comes with a very handy app so you can dial the sub in from the listening position using your phone, which is really nice and makes setup much easier. SVS offers a generous, 45-day risk-free trial period (including shipping both ways) and they also have excellent customer service just in case, so you’ve really got nothing to lose by trying one if it looks interesting.

https://www.svsound.com/products/sb-2000-pro

Here’s a link to their Outlet page as well…

https://www.svsound.com/collections/outlet-specials

At your price level I’d cross REL off your list at it’d be more of another woofer and not a true subwoofer, and while JL Labs is great you’re going to have to pay more and not sure in your situation it’d be worth the extra $$$ given what the SB2000 Pro offers.  Hope this helps, and best of luck.

First, look at the AM Acoustics Room mode simulator.  Keep your sub, speakers and listening location out of the lowest modes.

Your budget is a little small for the room, but may be OK.

https://hsuresearch.com/collections/true-subwoofers

The biggest issue IMHO is not which direction the sub points or ported vs. sealed but speaker vs. room.  I assume you will be using the Anthem MRX for room correction so it should help prevent the worst of the room modes from overpowering the sound quality. Go for the biggest baddest sub you can.

Another resource for you might be DataBass, the online subwoofer resource.