The first thing i would consider is condensation, imagine waearing rubber gloves and how sweaty your hands get. the gloves you can buy have a waterproof single layer goretex liner, not the welded fabric used in jackets because of the wear factor (and maybe the liner is much much more breathable. I think irts the same with boots with goretex.

Edit.

That is also to say that if you are handeling things the material becomes worn and this destroys goretex. If you line them with super breathable material that doesnt need waxing, and put a heavy duty permiable skin on them it should work. The goretex membrane used in gloves and boots does not need waxing so the gloves become wet only on the outside !