It's not limited to square waves. A sawtooth wave also has an instantaneous rise or fall time, as the case may be.
In the real world of music, a square wave would most sound like a clarinet, a sawtooth wave like a string, with bow spikes, a sine wave like a flute, and a triangular wave "flute-like" though none of these instruments are "pure of wave".