I always sleep with clothes on in the bag. Warm/mild weather I will just wear my Capilene 1 with a generic pair of boxer shorts and dress socks. Colder weather I may switch it to a Capilene 2 or 3 and a pair of generic wool bottoms.

It is very rare that I need to sleep in my down vest or a fleece during the three-seasons; but winter camping is generally a different story.