Only if there is the possibility of rain. I generally take my fly along regardless of the forecast; it weighs less than a pound. I'll pitch the tent w/o the fly in nice weather but the fly is nearby in case of a midnight shower. If you have a single-wall tent, though, you don't have a choice.
Loc: Portland, OR
For a quick overnighter with a strong forecast of continued fair weather, I could definitely see leaving the fly home. I don't know where you hike, but in my area I would normally bring the rain fly, because our mountains tend to manufacture their own weather and more often than not I am out for a much longer period than overnight.
If you do not need the fly (short trip excellent weather predicted) then, if mosquitoes or other creepy crawlies are not an issue, ditch the tent altogether!
Another option is fly without tent, using a tent footprint. Light weight tents nowadays weigh so little that the weight savings may not be worth the worry over the uncertainty, unless you can ditch the entire tent - that would be worth it.
Loc: Meadow Valley, CA
I'd only do that on a limited basis. My understanding is the fly is cheaper to replace than the tent body and you need to take into consideration UV exposure and its toll on fabric. Some materials are better than others. Higher in elevation, the more damage can be done. Duane
To clarify, the tent fly/footprint setup is for times when you definitely will need rain protection but could save some weight without the inner tent if bugs are not an issue. These are time when you would use the fly anyway, so it does not add to wear on the fly. The fly is going to wear out whether or not there is a inner tent under it.