I think Gerry really hit the nail on the head here. It really is a personal preference/what feels best to you decision. I carry about the same weight as you have indicated you normally carry (see my bio), and I have settled in on my McHale pack which weighs a tad bit over 4 pounds.

Previous packs I have used included an Osprey Crescent 90, an Osprey Aether 60 and a Granite Gear Vapor Trail. After using all of these other various packs I decided that the extra pound or two of pack weight was worth the extra comfort and superior fit of my McHale. The C90 was just to darn big and heavy (this was my pre-lightweight pack), the A60 was in general a good pack but just became uncomfortable over a multi-day hike, and the Vapor Trail just wasn't comfortable unless the total load was below a certain weight (25 lbs for me if I remember right).

People on this board seem to be all over the place as far as there are some who use 24 oz packs and carry 35 loads, and others that use 7 lb packs to carry 20 lb loads. I am of course over exaggerating a little, but I am just trying to make the point that you have to do what feels good for you.