Yes, it affects your sound, but aesthetics and room balance matter. What I mean is, shag rugs are nasty, so I totally understand getting rid of them, overall balance of the room can be affected by adding treatment elsewhere. For instance, would you add ceiling panels between you and your speakers? Do you need bass traps? Do you have enough diffusion?
Assuming the floor exchange was bad for the room, you can make up for it.