Your bivy isn't silnylon, but on silnylon, mineral spirits is safe to use because that's what dilutes the silicone sealer and allows it to be spread. I'd try pure alcohol first, as it seems to be least likely to damage the fabric. Try mineral spirits (used for paint dilution and cleanup) if alcohol doesn't work.

If that doesn't work, and you're stuck with the drawing, get a tarp. They're less phallic-shaped, and thus less likely to inspire drawings like this.