Unfortunately when you get a component on one end of the audio spectrum… like really efficient or inefficient… a really low power amp… etc. Then you end up committed to carefully swapping out supporting equipment. Then the next move is really hard to make… if it is to switch things out, then you are locked into the strategy.
Band-aides degrade sound… components not really built for your speakers degrade the sound. Your preamp and amp are world class stuff… but not ideal for your speakers.
OP, I would carefully think about the long-term. Would you like to be using these really high efficiency speakers or others like it in 20 years? If so, swap out your amp and maybe preamp to feed your speakers. If not, consider swapping the speaker for mainstream. Otherwise you can end up in a situation where just everything has to get swapped out.
While, it will depend on you. I would cut my losses and swap your speakers for those that sound right to you and are mainstream. But if the speakers are your cup of tea, then swap out your electronics. Lots of research will be required… otherwise you could easily choose an amp that suits your speakers but not the preamp… and maybe not the DAC… then you have a real mess.