I'll add my voice to the crowd here. WE take a sythetic underlayer that we sleep in, and then a shirt to hike in. And a nice big, lightweight fleece, and a rain shell. When it is really cold, we put on all four. We don't take down jackets, but there are days (cold ones) where we do put on all this stuff. And if it is really cold, we get into the tent and into our bags...but that's happened only once in the last 350 miles or so.

Keeping dry is essential. But when I am hiking, if I am more or less dry, I won't get cold because my body is generating heat. When I stop for the day---that's when it can get colder. But it won't get that cold that often.

In the end, it's all about personal choice, and personal comfort.

My wife gets colder than I do, so she carries and extra fleece vest.