If plugging one solves the problem and it sounds good, it is good! And to answer your question, No you are not hurting or degrading your sound by using one and not the other. All rooms are in need of some tweaks, or treatments, its kool. enjoy!
Regards,
Matt M