I've actually noticed this in both jackets and sleeping bags.

I'm a big guy. Thus I tend to find fit snug in most comercial bags. In every bag or jacket where there is any compression, I immediately notice chill in any place where the down is high-loft. With bulkier jackets, well, they don't compress so much at the shoulders, and I don't get chilled.

My solution has been simple. My fiancee and I sat down and made an 800+-down quilt that was actually big enough to fit me well. It is as big and heavy as a typical bag, but is made to be open beneath me. With no compression, it keeps me remarkably warm. I figured it was good for maybe 20 F, but slept in a tarp with a UL bivy around it down to around single digits back on New Year's day. BTW, this was under a tarp, not inside a tent.

Ligh-loft down has its place, and is very helpful for watching weight and bulk, but it does requires special consideration in its use.