I apologize if I repeat anything anyone else here has said, but no, 50's is not too old to start. I had a friend of mine who started at a similar age and really took to it. Back pain isn't necessarily a problem, depends on the type and how you deal with it. I'm in my 50's, have been backpacking since my 20's, and I periodically have back problems (hate it!!!), but oddly enough, for me backpacking itself doesn't cause it, except if I start a trip with it already bothering me, it can get worse.
The key I think is 1) work on your flexibility and joint strength (yoga is great for me) 2) be in shape from hiking on trails w/o a bp, 3) remember that gear has gotten a hell of a lot lighter over the years, and carrying a lighter load is the best thing for us, and 4) don't bite off too much at first, work your way up. The truth is that even going only a couple miles into the backcountry can be a glorious thing.It'll give you solitude, quiet, and a sense of accomplishment that is hard to get carcamping and dayhiking And then if it goes well, work your way up from there to longer more ambitious trips.