Am convinced after three seasons of use that the Sawyer cartridges cannot be completely dried. Even after weeks sitting in a sunny window I can still get water drops out of them.
My takeaway is it's very important to do the chlorine solution backwash before storing for the winter, then at the season's start, backflush before the first trip.
Some combination of bacteria and/or mildew is at work during that off period, clogging the works.
As to how visibly clear water clogging the filter in the field, it's likely to be bacterial biofilm. Have found field-backwashing can restore flow, even though it's a pita to do.
I didn't mention this in my post but the problem occurs between trips not after trips. The filter would flow well after backflushing it when I store it. When I took it out of storage after a few months, however, the flow was greatly reduced. So I think the problem has more to do with some aspect of the drying process (in my case) than floaties etc in the filter.