What should I upgrade first amp or preamp?

Karapuz,
Here is a different idea: If you live anywhere near Atlanta, you can spend a fraction of your budget and hire Jim Smith to voice your system. He can come over to listen and on the rare chance that he does not think a voicing session will improve your sound, he will not charge you his fee. I now have heard three systems that he has voiced, and the results have each been outstanding.
If you don't live near Atlanta, get his book or DVD and work on this yourself. You will be surprised at the level of improvement a well-voiced system will reach.
I know that does not answer your question about amp or preamp upgrades. It would be helpful to those trying to offer you advice if you describe what you think your system is lacking.

Karapuz,

I think you should try to define what it is that you are trying to achieve. I also think that those listening levels are higher than what I was expecting from your earlier description. Once you are clear on what you want improved, I second the suggestion that you contact a Pass dealer or Mark at Reno HiFi for a clear explanation about what you would gain from either the XP10 to 20 upgrade or the XA30.5 to 100.5 upgrade. Given that your needles are moving and your listening levels are not that low, I would lean toward the XA100.5.

I used to own the 100.5 and upgraded to the 160.5 because I was noticing the same movement in the meter and felt the amps could not quite drive my new speakers adequately. Mark advised me and his descriptions were spot on. Dynamics, impact, sense of ease, natural musical flow and inner detail all improved. I was really quite surprised.

On the other hand, the XP-20 is a fantastic preamp. And that upgrade should cost a lot less.