The Thousand Mile Summer, The Man Who Walked Through Time, and The Man from the Cave are all excellent. I don't remember much about The Winds of Mara (I think that's the title)...I should reread it.

In The Complete Walker, Colin mentions "The Territorial Imperative" by Robert Ardrey, which IMO should be on everyone's reading list.