If you try to further compress something that's already compressed, you will indeed get something that sounds "horriable." That's because the stuff that codecs strip out is already gone. If you go back to the original uncompressed file (if you have it), and re-compress it at a lower bit rate, it might still be acceptable to you. Sound quality vs. file size is a trade-off. Every listener has to decide for himself.
