Guys who drill holes in their toothbrush handle will certainly never be able to tell the effect on their pack weight, but they may find themselves with a toothbrush that has a broken handle!

To address the original question (or try to) I think there are a few 'inflection points' in total pack weight where carrying a pack a couple of pounds over that point creates a significant change in one's perceived comfort while hiking. For me, a total pack weight of 32 lbs feels like the upper limit of a comfortable carrying weight. Once I am over that weight, the discomfort of each added pound starts mounting rapidly.

Another inflection point, as my pack empties out on a longer trip, or as the weight I sometimes succeed in achieving at the trailhead for a shorter trip, is around 24 pounds. That weight, or below, always feels like easy hiking to me.

But as BZH points out, the key attitude is that weight matters, so any item I bring must justify its weight in the context of the hike I am planning. A few ounces will never matter on anything short of a thru-hike, but maintaining a certain rigor in evaluating every item, however small, is the only way to prevent hauling an overburden.