I'm not sure that this is true, but in our family children are too much care. My wife would not let me go alone with such a small child even to the lake for fishing, which is 30 minutes away from the house by car. But also in our family, it's customary to respect other people's hobbies. I don't propose my wife to take the child to the spa with her friends, just as she doesn't mind if I go hiking or fishing with my. Maybe you really should wait until your child grows up to go hiking with him. But now you can go hiking alone, your husband cannot forbid you this.