What's a good amount of time to have my preamp unplugged before I swap out tubes?

You don't ever risk electrocution in removing a tube after an amp or preamp has been turned off.  At worst, if you somehow touch a capacitor that has not been discharged, you could get a local, nasty shock.  But, if you are just pulling a tube, you won't be touching anything that can deliver the a shock.

It helps to wait just a bit to allow things to cool down.  This prevents you from getting a burn or accidentally dropping a tube to hot to handle, or perhaps jarring the innards of the tube while they are hot and more sensitive to movement (not a likely scenario anyway).

The only concern for the gear is that you may not want to turn something off and then turn it back on so quickly after first turning it off.  Some gear reacts to such treatment by making loud noises through the speaker and causing a rectifier tube to flash.  This can be avoided by leaving the gear off for about a minute or two before turning it back on.  

These are very handy when the tubes are too hot when replacing!!

I wait a minute or so and grab using a microfiber cloth to insulate my fingers from heat and I don't get anything from my fingers on the tube.  I also use a microfiber cloth when I insert the tube to mitigate contamination.  

Aside from getting burned, there is no reason why you should avoid contact with the glass envelope of a tube.  Tubes do not get so hot that oils from your hand could lead to failure.  The best reason for not touching the glass when changing tubes, involves tubes with a base (i.e., not entirely glass)--you do not want to pull tubes by the glass because that might lead to separation of the base from the glass envelope.  

The whole idea of making a fast swap suggests some sort of quick comparison of tubes.  Of course such a comparison is not entirely practical because it will take some time for the new tubes to come up to good working order (a few minutes).  While it is important to get tubes you like into your gear, one should be mindful of the wear and tear on tube sockets that constant swapping causes.  An old technician told me that most tube sockets are good for about 30 in and out cycles before the socket needs re-tensioning or even replacement.  In the old days (before neurotic swapping) that would mean 30 or more years of service).

Room temperature.