My experience has always been that if I rely on fish for food, I never catch any!

As others have noted, the caloric value of fish is pretty low. Fine for those of us who have extra padding to lose but not for those with the lean and hungry look. Consider the fish a supplemental treat, in the same class as wild berries.