I'm not suggesting you can't do without a bag, people do it all the time either intentionally or not. I could get along fine in pretty cold conditions if I just had my thermals, insulated pants and my big parka (the red one in my picture), down booties, balaclava and gloves or mitts and a mat to sleep on.

My point is why bother. I'd much rather be in a bag and be comfy than wrapped up in my clothes all night.