If I am doing actual backpacking i.e. 10+ miles per day then I generally will not bring my fly rod mainly because of the reasons previously mentioned-- you can just never guarantee that you will be near an adequate fishing source.

Now if I am just backpacking in and setting up camp for 4 days and 5 nights near a fishing spot, then I will plan to eat a fish lunch 3 of those days and for dinner maybe 1-2 of those nights.

I wouldn't rely on fish to sustain my body if I was putting in serious milage every day.