I mostly left the sleeping bag inside the bivy, and just stuffed the whole package into the bottom of the bag. (I usually zipped the face cover closed, to keep condensation or other moisture on the outside of the bivy. Rain was never an issue; I always used a 6x8 or 8x10 tarp, so the only moisture was the occasional light stuff that blew in. )
The pad would be inflated and inserted at camp, the removed and deflated for hiking.