There are no "must have" records. The only records you need to have are the ones YOU like.
I have a weird taste; one of my top album will be considered beyond bad by most: The Ex: Blueprints for a Back Out. It is experimental jazzy punk. I like the band so much that I named a species for them: Depressizona exorum (marine snail from Easter Island).
I also like the one and only record by Rites of Spring (metal-hardcore punk crossover). More recent band is Bedless Bones out of Estonia. EBM/electronica to ambient. One out on vinyl, the rest only on CD/digital.
Razzia's Tag Ohne Schatten is exquisite, IMHO, if you are into hard and rough German punk.
Virgin Prunes "If I die, I die", classic over the top post punk. If you like the heightened theatrical aspect, then Bestial Mouths may be up your alley. Also very much an acquired taste. Similarly, Spear of Destiny's Grapes of Wrath; laster SOD unfortunately cannot come close to the first.
If you like stuff that does not fit into any category, try Snow Ghosts (Neofolk with synth), earlier Swesor Bhrater (Sang lié, Sick Rose: also neofolk) or Mission to the Sun (indescribable gritty lofi sound out of Detroit, on Felte Records).
Consider spending some time on Bandcamp, listen to various styles, follow links and just explore. That is the fun of it all. Early on (early Y2K), I did some random streaming (pandora) to get a toe-hold into newer music, but now I don't stream anymore as Spotify etc. exploit artists, IMHO.
Another one I love is Jordi Saval "La Lira d'Esperia II: Galicia". Renaissance Rebec and tambourine music from Spain. Only on CD. Have not found any morsel of the CD on the web to listen in. Just in case you are into Early Music.
my 2c.