Help with speakers, equipment, satisfaction

It sounds as though you've pretty much decided that you don't prefer the Vandersteens to the ML's. Yes, it's strange that the bass contest seems to go the ML's over a speaker that is extensively engineered to provide SOTA one-box bass, but there you go. The thing I immediately think of is that with so much adjustment possible in the bass, maybe you haven't got the 5's properly dialed-in yet, but you seem to think that you and your dealer have this covered. I don't think your variety of partnering gear can be blamed, and you've tried the Vandy's out in the bigger, treated room, so what's left? I think the "A" upgrade of the 5's includes a better bass amplifier, but how can you be sure you'll get everything you want top to bottom if you spring for it? Obviously, even though the low bass isn't handled by the main panels, an ESL is going to give a much different presentation overall from a dynamic box speaker, and your allegiance seems clear. (Or could the Vandy's actually be telling the truth, but you're underwhelmed?) Whatever the case, maybe you should look into the newest ML's such as the Prodigy or Odyssey instead, which are supposed to be even more accomplished in the bass than the older ReQuests, though this would probably mean having to do the acoustic treatment again in your listening room if you want comparable results. I live in the area - email me if you want, but I think you already know what I think you already know... ;^)

For one, there is a MASSIVE acoustical difference between your two rooms! The much larger space in the ML room is going to give you a much easier time at getting speakers and chairs properly set up for even bass response. What I've found is that inexperienced audio enthusiests get LUCKIER more often with things sounding better in LARGER ROOMS, than much more tricky and difficult smaller rooms! The Bass modes are more evenly spread out in the larger acoustic space, and you're more likely to get a better spot for your speakers and or chairs, chance wise.
I've done a lot of rooms at both extremes. Again, by far you have an easier time of things in the larger acoustical space. That doesn't mean that careful set-up isn't mandatory in a larger room, as you can still easily get things in bad spots resonse wise.
Have you measured your bass response with test tones and a meter yet?(most never do). You probably have either your speakers in the smaller room placed so they aren't coupleing well with the room(maybe boomy on bottom, and week through the midbass...and peaky elsewhere up the bass spectrum), or your seats are improperly located, or both!
You probably have boom and exageration in the lower bass with your speakers as well as holes and lack of energy in midbass, and uneaven on up in the bass in the small room set up.
This really isn't rocket science when it comes to measuring frequency response. Show me a set up with reasonably flat frequency response, and you'll have excellent bass performance overall! There are other issues, such as reverb time, but overall speaker and seating placment are your main challenges of hand. Get those right, matched with your more than capable speakers, and you'll have pheonominal potential in the bass response!
You might find that proper placment of seats and speakers in your small room, along with some EQ'ing of the bass(bi-wirable Vandies?...use the bottom posts with a separate amp and a good parametric EQ!!!), will yield you tremendous results in bass response. If you don't know what you're doing, this can be challenging to receive best results.
My recommendation to anyone looking for best results from their sizeable invesment is to get some professional consulting (www.rivesaudio.com, for one)help! When you're done, you'll have a much better grasp with with all this, and your system will sound 100's of percent better!!
The room, set-up, placment, acoustics, calibration, and tweeking are easily more than half the sonic equation!..as you can see/hear.
A quick trick(although not the last word) to finding the best sound (in the bass) from your speakers however,is to place one speaker's bass woofers/port location where you're ears are going to be in the listening seat is(good placment of seat is just as critical), and play some music with a good bass bass beat(I use th 5th song on the Tracy Chapman album), while you listen down where the speakers woofers are likely going to be located in the room ultimately! If you can move around on the floor where your woofers are going to be(depending on speaker choice and needs...mini's will be up on stands), and listen for the best bass response, you'll find the best location! You can then both measure the bass response(which should be close to flat when it sounds best), and then replace the speaker where your had was, and go back and listen from the chair! You'll have the same good bass response you heard where your head was before where it is now!
There are considerations, such as whether or not you're playing the speakers "full range",or playing them as "small" with a sub woofer in an HT system however! There are different approaches for different scnearios. But still, even if you're playing a woofer up to a certain frequency range, or whether your smaller speakers are playing down to, say, 80hz only, you can place the speakers using the above described method within their effective ranges! In a sub/sat system, getting proper coupling at the crossover section is critical, and not often pulled of so well! Get it right however and you'll have excellent "full range sounding" results! The trick is to get proper balance,phase and coupling from the sub and sat's respectively!..so they're sounding like a well balanced/integrated "unit!"
These are some helpful suggestions for bass response, and better than what most ever end up with (especially with multiple speakers) by simply guessing or assuming!
My bet, again, is that you have problems with both speaker and seating locations in that smaller room you're dealing with! I've done many rooms of this size(approx), and I know what you're dealing with if it's closed in at those dimmensions.
Do some experimenting, or consult a pro. That's my suggestion