I use a VBL too, but it consists of non-breathable rain jacket and pants worn over just my polyester base layer. That way I can still add insulation layers over the rain gear without getting them full of moisture. It is important to remember that whatever is inside that VBL is going to be soaked by "insensible perspiration" from your body. (That's why the polyester base layer, which doesn't absorb moisture.)

With this setup, I have taken my sleeping bag with a comfort rating of 24*F (I am female, elderly, and a cold sleeper, so find the "comfort" rating just barely warm enough) to 15*F and have been quite cozy. I'm sure I could go down to 10*F with a warm enough sleeping pad. The VBL also keeps my body moisture out of the sleeping bag insulation, which is really important in below-freezing weather. I don't, however, use the VBL unless it's going to be a frosty night.

My setup won't work with breathable rain gear, though. Like a number of others (Andrew Skurka is one), I have given up on so-called waterproof/breathable gear, having found it neither waterproof nor breathable.

Some folks can't use a VBL at all; it depends on your individual physiology.

I have also found that the draft collar on my sleeping bag really boosts the warmth and lets me loosen the hood a bit to let my exhaled air escape instead of soaking the hood opening. For those not having a draft collar, I'm sure that extra insulation worn around the neck to keep the cold air from going farther down would help, too.