When you feel that you need to replace the room or the entire house to get the right sound. That's the point, I think, when one should stop upgrading. At least for a while.
Other than that it is highly subjective.
Other than that it is highly subjective.