What do you say to your audiophile friends who say you just listen to your equipment more

My friends are smarter than that. If anyone did say that more than once, they likely wouldn't be invited back. I don't need mindless negativity. :)

I'm in this for the gear as much as anything else, and no one can tell me I'm doing "audiophilia" wrong.

I'm a System Builder, not just an audiophile. Been doing this for about 35 years, since my early 20's. I have chosen to focus on a different aspect of the hobby, one that involves a great deal of variety of sound through building systems, which is one reason why I have some speakers for sale at the moment. I have allocated my time, money and energies to an alternative aspect of it that gives me far greater pleasure than experiencing the same system every day.