You are quite right - it doesn't. But backpacking - putting one foot in front of the other until darkness descends and then making camp - repeat as necessary- can be undertaken in a variety of venues - some of which are more intricate and demanding than others. Backpacking interfaces with mountaineering, canyoneering, and caving, among other pursuits, which require more technical skills. And then there are inadvertent mishaps, either your own or someone else's. Sometimes we like to push limits, and sometimes we just like a nice relaxing hike and a pretty sunset.

I can really relax when out with capable companions; when otherwise, I need to be more vigilant and careful, taking more responsibility for the group. I think this is some of what the OP was addressing.