Not all capacitors store energy for transients. Those big ones in power supply - yes. They also act as part of filter reducing "ripples" in the DC which remain after AC/DC conversion.
Now, small caps in the amplifier are NOT for power storage. They serve many purposes. For example, isolating one cascade of amplification from another. Capacitor passes AC but not DC. Music is AC (not to be confused with AC mains). Another purpose is to improve stability at high frequencies. Some may not be even part of the amplification and rather operate buttons and such.