What kind of pack are you using? From the way you describe it, it almost sounds like you're using an external frame pack. If so, you're approaching it right; you'll definitely want to use a stuff sack for the bag, though.

If it's an internal frame pack, the sleeping bag should go inside, usually at the bottom of the pack. A stuff sack isn't strictly necessary (I just stuff mine in loose, so it fills all the space) - but if you use one, you might want to use a fairly large one, so you can mold it into the bottom of the pack, too.