Either can be a great a experience. However, you're missing something important with headphones. Headphones completely isolate left channel from right channel. Stereo is engineered so that the right ear hears some of the left speaker and and vice versa. That's why they use headphones in the engineering process...to isolate and determine the degrees of blend.
You're missing some of the imaging experience with headphones. That said, headphones naturally eliminate much of the music/space interplay and will be easy for most to enjoy music. It just eliminates all the variables.
But if you're willing to make some big concessions to the room...and most cannot or will not...you will be rewarded with a sense of scale and impact that headphones can't quite reach.
My current room cannot eclipse a decent set of headphones. But I'm not giving up because my last room surely did.
Rooms are tougher than headphones. This is my experience.