I think you'll find that you'll be carrying, not wearing, a lot of that extra insulating clothing while you're actively hiking, especially going uphill! You want to avoid getting sweaty if at all possible, especially in cold weather when wet insulation = hypothermia! And whether worn or carried, the weight is still on your feet!

If you're going for a quilt, you want a pad with plenty of insulation. For a 20* quilt, you probably want the R value of the pad to be around 5. This is also true for a sleeping bag (the EN13537 criteria for testing 20*F sleeping bags puts the "sleeping" dummy on a R5 pad). I'm still looking for the perfect sleeping pad and have about given up. My Exped Downmat UL7 is plenty warm, but it appears so fragile that I'm nervous about turning over, and for me it's also not quite thick enough. When I have a choice (i.e. it's not too cold), I still use my old pad (POE Max Thermo) I bought in 2006. It has long since been discontinued--even the company has disappeared--so there's no point in my recommending it.