The main reason I prefer night listening is the darkness. I want my room to be as dark as possible so the speakers disappear. Even the soft glow of tubes lights up the room more than I like.
Secondary reasons are work and other obligations during the day means night listening is required. Less traffic, less ambient noise, and less distractions are experienced at night as well. Deeper into the night, around the three hour mark of a listening session, is where things really begin to sound fantastic. By then the components are fully warmed up and my hearing has relaxed and opened. Finally, we are conditioned to a degree to equate night time with fun time. The evenings are when dinner is eaten, movies are watched, parties are enjoyed, and drinks are consumed.