I suspect it's something that was coined to make folks feel better about their choices. I have no doubt that since the driving cohort is massive compared to the backcountry adventurer cohort the latter has a much higher fatality rate. I'll guess all backcountry activities get lumped together when considering fatalities, but (luckily) backpacking will never be as hazardous as, say, big wall climbing or out-of-bounds skiing.

FWIW I've known more people killed in the mountains than in car accidents which, now that I think of it, is pretty amazing.