Leave it on?

In most cases this is just picking nits. Well-designed gear that is relatively low in idle power draw (preamps, class D / G / H amps) should have NO problem either way. Do what you like. Gear that is high in constant power draw, are therefore heat dissipation - tube power amps, class A and high bias AB solid state power amps - should really be kept OFF when not in use. In my experience, hotter components are also more likely to sound good quickly after warmup - simply because they hit thermal equlibrium quicker. SS is always slower to warm up than tubes.

@soix 

Yes, definitely ask the manufacturer on that. The more sophisticated (expensive) balanced tube preamps I’ve owned (ARC Ref 6, Rogue Hera, VAC Master) have relay-timed outputs: the outputs are not "live" during a warm-up period, and are first disengaged on shut-down. These have been OK when I’ve inadvertently violated the usual start up / shut-down order - no thumps. BUT I think for general "gear hygeine" it’s still smart to stick to the usual order, as much as possible. But also, these days, with all the protection options available, I think it’s very hard to justify manufacturing any preamp that could cause an awful THUMP in speakers!