"Your brain cannot tell [t]he difference between the direct and reflected sound if the wave path distances are virtually the same."
This is a wavelength-dependent argument and breaks down at intermediate frequencies. The reflections can come back from the back wall and interfere out-of-phase with the direct sound (to varying degrees) creating a strange frequency combing effect right in your ear. This becomes important for wavelengths of the order of the deviations in the round-trip distance from ear to rear wall and persists down to wavelengths of the order of your ear size and the distance of uncontrolled head motion. So, if you are about a foot or so away from the back wall then this combing effect will happen for frequencies below about 1,000 Hz on up in excess of 10,000 Hz. This can be very disconcerting.
One way to see if this is a problem in your situation is to cup you hands over your ears so that the opening is pointed directly at the speakers. This will allow you to hear more direct sound at the expense of the reflected back waves.
I use a panel of three-inch thick acoustic absorber behind the listening position covered with a 1-inch pile tapestry (read carpet).
It is also possible to minimize this effect with diffusers. There are commercial products that use both diffusion and absorption. They look like a 3-d foam model of an urban canyonland.
