How important is spending time with your gear?

In another topic we're talking about digital input speakers, and it got me thinking about something entirely different. 

How important is it to spend time physically close to your gear, vs. enjoying it's output?  If you could have your gear in another room, or closet, and you were left with just your speakers with no audible downside would you do it?  Would you put your gear away and enjoy the empty space or do you need the physical closeness?

Clearly turntables make this a challenge, and there will be some poopy heads which don't get the question or can't stretch their imagination but for those who can, would you?