Amp making noise on shut off

Is replacing caps a factory repair job?

Any competent technician could do the job. Is the amp in warranty? If yes, I'd send it back for repair.

I've seen solid state amps do this too. This is a bit more concerning than it is in a tube amplifier; it may mean that an active device is failing. If this is the case I think you'll find that it gets noisier to the point that you hear it when the amp is on- at which point it should be serviced, as the noise could be the result of an oscillation. I wouldn't worry about it other than that.

Meaning: nothing to worry about. Some tubes will do this and its not an indication of failure or sound quality.