Question--are you trying the packs fully loaded with your gear? That should be the critical test--load the pack with your gear (remember to include the equivalent in weight/bulk for food and water) and do a couple hours' hike around the house--or around the neighborhood if it's a nice day and you can keep the pack clean. Of course, don't remove the tags.

The most important aspects of a pack are fit, fit and fit! Pack fit is almost as individual as shoe fit, so what works for one person may be awful for another. I won't recommend a make or model of pack any more than I recommend shoes! For me personally, my hips and Osprey hip belts are completely incompatible, but that may not be true for you.

As a general rule, if you are an in-between size, you are better with the larger size. From your description of your S/M tryout, it sounds as though the size decision has been made, assuming that the larger size is comfortable when loaded with your gear.

If you are "on the fence" with the larger size, and since the smaller size is definitely too small, I strongly recomment trying a few other makes/models to see if there's something that works better for you.