Beryllium dust is deadly but as no one will be subject to beryllium dust from a set of headphones I think you are good. I don't suggest that anyone should extract the beryllium diaphragm, drill it to create a cloud of dust and then inhale deeply :) other than that you are safe.
I heard the $4000 Focal Utopia Headphones yesterday
Very nice. Very expensive. Beryllium drivers. Reviews a bit hyped I think. Who would pay that premium otherwise? Not sure I could live just with any headphones still.