If it's 20 in the morning, you'll need a fleece or puffy for warmth. I just use mine to sleep in, too. The bivy might be a gain, and probably is, but extra sleepwear may do the same for no additional weight. I make sure my winter bags are roomy enough for the extra clothing and neither the puffy or the bag's down is compressed. 20 degrees is a big swing, I'd be very hesitant about gaining that much of a comfort factor out of the bivy alone. A down beanie and a puffy, on the hand, adds a significant amount of warmth. I've gone through many permutations of sleep systems trying to stay warm and lightest possible. Probably more on the way, as my wife and I are starting to section hike the Long Trail in Vt. and a lot of it will be done in the winter. If you do any testing with the bivy, I'd love to hear your results.
_________________________
Charlie