I like to separate the shell and insulation functions. The outer shell is typically nylon or polyester with DWR. Depending on temperature, I add or subtract insulation layers underneath. I'm partial to merino wool for that, but poly fleece should work fine (some would say better).

When it's too cold for the woollies, I go for a garment with pile insulation like the Patagonia Micropuff pants. They are anything but cheap, however. Down pants might be the ultimate, but I don't tend to go out in weather where they would be required.