Those extra channels make a world of difference. Dropping from 5 to 3 would be huge reduction in the home theater experience. Turn off the rear surrounds for a while.
My primary interest is two channel audio, and I am fortunate to have two systems (see my profile). I used to own Olympica.
Either upgrade the left and right channel… I assume you are using those only for two channel audio or leave things as they are.
Long term, if there was a way to separate the systems you would be doubly rewarded. I find that overall the quality level of home theater is different and fine nuance not important since the video is so distracting. You could plan to extract the best components for your two channel and replace with home theater level components.