If the noise is constant whether music plays or not, AND it’s not coming from the speaker... that eliminates pretty much everything.
It could possibly be a buzzing inductor, though that’s kind of reaching. By nature all inductors buzz so they’re potted in resin or glue to quiet them down. The dampening material can fail with age.
If that’s what it is, it’s completely harmless.