I use 6.5" drivers in my mains and they do put out a decent amount of bass, but I’ve also integrated with real subwoofer and the result was amazing.
The easy, correct way to go from here is to go as big as you can with the sub BUT use appropriate EQ tools like miniDSP or get an SVS sub with built in EQ. You are right that subs can create problems, which small speakers don’t, but a good EQ can clip the big mosters off so the sub can work correctly.
If you are unwilling/unable to apply EQ, then leave your set up alone as-is. If you ARE willing to measure your sub and clip the excesses then I encourage you to go mainstream with a 12" - 15' sub.