Well. all I can say is try it out and see what happens. I personally don't like the idea of sleeping in my clothes, but if that works for you, then go for it. One thing I don't like about your idea is the possibility of getting the inside of my bag any dirtier than it would otherwise with just me and my base layer inside it. I've tried sleeping with a jacket on and find it really annoying since I tend to thrash about a bit before I fall asleep.

There really is no one answer that is perfect for everyone, unless, of course, you are Douglas Adams, in which case the answer is "42." And yes, I will explain that to anyone who asks.