I wear them almost every time I am walking (except of course in town). I have a number of sets.
The ones from China that you get very cheaply work as well (mostly)as good ones, but they are fragile and often do not last an entire summer. I give the cheap ones a coat of waterproofing just above the ankle.
I am broke however and you might be happier buying a higher end set that is more durable.
The only time I do not wear them is when it is guaranteed to be dry everywhere and I also do not need lower leg protection.
In the Northeast, that is almost never.