Moetbrut, they are compatible in the sense that neither one will cause any harm to the other. They may not be the best match sonic-wise but I doubt it is the worst either. I have a Denon 5800 (which has the same amp section as the 5803) which I used to drive my Soliloquy 6.5s with and the sound was not that bad. A little bright, maybe, but that could have been addressed by using a little darker sounding speaker cable than the Kimber Bifocal XLs I had. Acoustic Zen Satori would probably have worked fairly well. The Vandersteen 3A has a similar tonal signature to the Soliloquy 6.5 so it could probably be made to sound decent with the 5803.
I was seeking a little bit better sound quality than the 5800 could give me so I added a Rowland Model 10, which was driven from the preamp outputs of the Denon. This was a better match tonally, plus the Rowland is an all around cleaner sounding amp than the Denon. The Rowland is one of the better SS amps available, and certainly produced a worthwhile improvement, but it was over twice the cost of the Denon. I auditioned many cheaper amps before I bought the Rowland and didn't think they offered a significant enough increase in sound quality over the Denon to warrant the expense. The amps in the Denon are comparable in sound quality to separate amps in the $1000 to $2000 range. Don't believe the receiver-haters that say any separate amp will be any improvement over the internal amps in the Denon. It simply isn't true. You may achieve a different sound, but it won't necessarily be a better one. In my experience I had to spend quite a bit before I thought the sound quality increase was worth the trouble.
If you are considering the 3A/5803 combination because you want a primarily home theater setup I think you will be fine. If you want to use it for primarily music then you will probably be happier with some good quality seperates. I'd look at Rowland, Pass, and Ayre for starters. If you want to consider tubes, I've heard the Audio Research VSi55 integrated amp with the 3As and it was a very nice match.