You may think at first that you're hearing a more transparent signal but a preamp properly matches the impedance between stages and provides more muscle (better dynamics) in the long run if it's a fairly good pre which it sounds like you have. You will get more dynamics and you won't be as susceptible to roll-off of high frequencies. The pre-amp is more than just a handy switching device, it provides an easy load for whatever you plug into it and outputs a lower impedance which will make the amplifier happier than if you just plug in a source device.
Should I eliminate my preamp?
I have been using my Audio Research tube preamp and Bryston amp to drive Magnepan speakers for years. Recently I added a Oppo blue ray player to my system and connected directly to my amp using the balance cables. The reason was to eliminate the signal having to go through another piece of equipment before it hits the amp. Am I wrong or what am I missing?