I'm late to this party, and there's a reason: mostly, I could care less what others wear; I'm just glad to see them out there enjoying the woods. Personally, I use drab colors (grey, green, tan, brown) when I can, because I like not to stand out. My colors tend to be one-per-item, but I'm not sure that's a whole lot different than camo, which is multiple colors mixed into each piece.

The only side issue I could think of is one of perception. Just as having clean, high tech gear or wearing certain brands of clothing "brands" you as a "yuppie" or "gearhead," wearing camo can brand you as "paramilitary" or "survivalist." But don't let that dissuade you from wearing camo. Sometimes, being branded will cause people to leave you alone (and I've found those are often the people you most want to leave you alone.) The folks worth meeting will talk to you regardless of how you're dressed.