I don't know if you have a Dunhams near you, but there are a couple where I used to live and they have a lot of sale racks. I stocked up on base layers and got a nice medium weight moisture wicking longsleeve for $15 (along with some pants for the same price).

I would hesitate to buy something from REI without looking in your local store (although $21 isn't bad). I am 6'5" and 180lbs (long and skinny), and I don't usually have too much of a problem finding shirts that fit. Quite often the name brand stuff is over priced (but not bad stuff).

Basically to reiterate what the other guys said, stay away from cotton if you are going to be doing anything remotely hardcore. Honestly, 50 degree camping is right around the sweet spot, in my humble opinion. Once you get moving around, it feels perfect. You don't really need to start worrying until it gets real cold. I have camped in -20 windchill...that was a little chilly. I am sure there are people on here that have done better than that though.

Good luck, have fun with the cold weather camping