I am not a guru, but although I’d think experiment with both if you have the outlets available (or enough speaker cable) I’d THINK having the sidewalls but having whatever is in the dining area as a rear wall would be the better option.
I’d THINK that with this option, your reflection points to the side would be equal, and if they were not, I would also THINK that your left/right balance would be skewed. As far as the dining room where the back wall would be, at least there will be stuff back there for diffraction and I would also THINK it might be advantageous to have that as opposed to a reflection off of a back wall that was closer.