I agree with those that say you need to live with the new CD player for a while. When you switch back to the cheap player, you will notice the change more. It may be something big, or as simple as listener fatique from the harshness of a cheap player. Most likely the change will be in the little things that add up to a lot.
Secondly, I agree with Bob on cables. $25 cables are a choke point. It is like putting cheap $25 tires on a Porsche. With cheap tires, a Porche will perform like a Chevy Cavalier.