In the "old days" when there were not sufficient sizes in packs, I always had a pack that was too big for me. It just makes it more uncomfortable, not impossible. If the hip belt fits, then the length is less of a problem. If the hip belt does NOT fit, then the pack is pretty much useless. A lot of packs have removable, interchangeable hip belts that can be replaced with as smaller one.

Experiment with packing it differently. Keep heavy items near your back and not above your shoulders. Try it for a while before you totally give up. It may not be as bad as you think.