I agree, it's silly to have one set of gear/clothing for hunting and another for backpacking! Just ignore the comments!

The one thing I'd worry about is being seen when you need to be seen. A few years ago while driving home from a hike, I came perilously close to hitting someone sitting at the edge of a road in the shade. I was dodging potholes on the very rough road and didn't even see the guy until he moved! If you need to road walk, you might want to add something bright-colored (and reflective at night or dawn or dusk) to your clothing.