In my experience, wearing a pack over a down jacket will reduce the loft; clearly where the pack rests and where the shoulder straps ride, less clearly through tension on the outer fabric from the shoulder straps compressing the underlying down.

For me, the only time I wear my down jacket is in camp. Even in moderate sub-zero weather, if I am exercising at all, I sweat; I'm sure that the sweat condensing in the outer down layers reduces the loft and I also think this is progressive over several days. Things may be different at temperatures nearer -30 to -40 though.
_________________________
May I walk in beauty.