I used to do water quality testing. I know we had certain non-glass sample bottles that were designed to be non-reactive. There also are specialized plastic bags that do not "bleed" plastic into samples (if you are also worried about carrying backpack food in a bag). If you have a water quality laboratory near you, visit them and see if they will sell you some bottles. You can find suppliers on the internet, but many will not sell a small quantity to individuals.

I feel the risk of a plastic water bottle is worth the trade-off to have a lighter pack when backpacking. I think if you buy high quality (like Platypus), not some off-brand knock off, the risk is a bit smaller.