Can using a bad tube hurt an amplifier?

Yes and no. The tube can just fail and stop working with no other symptoms, or it can fail and cause a fuse to blow, or burn up a resistor, or destroy an output transformer. It depends on the amp. For what it's worth there are millions of tubes and tube amps in use and no major incidents.

It can cause serious damage ,  I wouldn't risk using a tube if zi suspected it was bad. 

Tube amps will use either an automatic  bias scheme where the amp will maintain the bias voltage for the tube or is is manually biased periodically by means of an LED , built in meter or multimeter.   

If you suspect it's a bad  tube, replace it.    Usually a bad tube will just blow a fuse or take out a bias resistor  ut it can cause catastrophic damage. 

@emergingsoul 

Which tube amp do you own or are considering purchasing?

When you mention "and then voltage changes over time gradually" are you referring to tubes going out of bias or actual voltage changes from the power grid?  If the former, it is imperative that a tube amp be easy to bias or self-biasing, which most newer offerings are.  

A bad tube can definitely damage an amp or preamp.

and speakers....they can damage those too. I always use a broom handle when I'm turning on my tube amps...