... how important is Damping factor in determining what amp to buy

The VTL  data sheet states that the lowest DF setting is the most "natural" sounding. This seemingly implies that the less negative feedback the more natural it will sound. Larryi this correlates with your observation mentioned above. 
Charles,