My personal conclusion is that what is generally accepted as an "audiophile" includes the behavior of going to great time and expense to achieve good sound for whatever purpose, and I'm therefore not an audiophile either. Because I talk about music and equipment more than many of my friends, they readily identify me as an audiophile, but the truth is that I like listening to good music with a good system, I've invested a fair amount in a good system, I read the usual magazines regularly, but I'm only going to pursue upgrading if I'm convinced that it's going to be an eye-opening experience - I don't spend a lot of time tweaking or changing the system to experiment, etc.
So, my theory is that there aren't many females who are audiophiles, but there are plenty who love music and many of those that appreciate good sound. The visibility of the split is that, while they appreciate good sound, they don't choose to spend their time reading the magazines, discussing the pursuit of good sound with their friends or spending a lot of time on message boards such as this one. Males are more likely to do so in audio for the same reasons they're more likely to do so in sports or autos or other similar pursuits.
The part that never ceases to amaze me is how few people change their perspective when hearing a really decent system, male or female. Most hear the difference, but go right back to listening to a cheap boom box without minding a bit. I would think the exposure to something that much nicer with something as fundamental as music would change more people, at least to want something adequate.