When I do a long combined road trip/backpack trip, my biggest fears are mechanical failure of my car at a remote trailhead. I feel totally comfortable in the wilderness on my own, but I am a terrible mechanic. Do be careful where you car-camp. The closer to civilization you get the more dangers. I like FS or BLM campgrounds, off the highway enough to get away from the noise, but with camp hosts. I always introduce myself to the camp hosts. I do not even to short naps at rest stops. I really do not feel comfortable at these. If I camp at a "dispersed" FS site, I find one that is not visible from the road.

As for meeting weird dangerous people on the trail - I spend very little time on a trail. Last trip I did 8 days off-trail and never saw a person. Yes, I have run across a few eccentric folks on the trail, but nothing "dangerous". (But then, I suppose people who meet me on the trail think I am eccentric too).

I have hiked alone in grizzly country. Not preferred but I accepted the risk. I took bear spray. By the way, if you use bear spray be sure to practice shooting it.

I think one thing that is helpful is to read some statistics on what things are really dangerous. What is the probability of getting struck by lightning vs mugged in the wilderness? It may make you and your family more comfortable if you (they) knew the facts.