That's totally valid. I'm not sure where you're located, which is a factor as well. Unfortunately, physical hifi shops just aren't as common these days. I'm in DFW so I do have the luxury of being able to find Rossos, and many other brands, within driving distance to audition in person. However, there are still several brands I would love to consider that would require a flight to audition. Fleetwood, Triangle, Tannoy and even several of the brands you mentioned: Wilson Benesch, Franco Serblin, Vimberg, and Avalon would all require an out of state trip to audition.
This is true for most people in the US. If you want to audition high end speakers, you really only have a few options. 1) stick with options in driving distance 2) Travel to an audio show and hope the brands and models you want to hear are represented or 3) order blindly based only on reviews and hope for the best 4) take a dealer trip.
That experience isn't exclusive to Rosso. It's the reality for the vast majority of the higher end hifi industry. Most places simply don't stock speakers that retail over $10K. And outside of a major cities, most places don't stock hifi gear at all. I know you said "fly all over the US" but that really isn't the case. If you plan a trip to the LA area accordingly, you can find most brands available in the US there.
Also, if you were just talking about the Elbas, then the price of a trip probably isn't worth it. But if I was shopping in the $30K range - I think a trip to LA is pretty cheap insurance if you're not satisfied with the options available in your area. Not exclusive to any one brand. It's just probably the best place to hear the largest number of brands outside of a loud and crowded audio show.