Why Don't More People Love Audio?

Can anyone explain why high end audio seems to be forever stuck as a cottage industry? Why do my rich friends who absolutely have to have the BEST of everything and wouldn't be caught dead without expensive clothes, watch, car, home, furniture etc. settle for cheap mass produced components stuck away in a closet somewhere? I can hardly afford to go out to dinner, but I wouldn't dream of spending any less on audio or music.