On the other hand, we only have so much time, and there is a universe of more accessible music available.
millercarbon describes my situation and approach very well. I'd add that repeated listening to something not to my taste is unlikely to warm me. When my wife was in her graduate music program, I must've listened to a Scriabin (and one other I can't recall) piece on the piano hundreds of times. It was the proverbial "Fingernails on a blackboard" to me and I just don't warm up to that. More power to those who can acquire a taste.
I have a better record for acquiring taste in food, but if it's on the menu, yeah, the lamb. Or the calf's liver.