Any water container filled with a couple of liters of water won't even come close to freezing overnight if the temperatures are simply 'below freezing'. The very worst you'd experience would be a very light skin of ice on the periphery of the bladder and that would probably require something closer to 20 degrees overnight. Your water filter is better off sealed in a plastic bag and put inside the sleeping bag with you on nights you are worried about sub-freezing temps.

I can't say how safe or comfortable a pillow it would make. I just use some clothes items inside a stuff sack.