I use fleece to layer, and wind/rain jacket over that. I have a nice poncho if it is really coming down, I carry it always and use as a ground cloth.

My 3 season layers weighs anywhere from < 1lbs to 4lbs, depending on where I am at. Fleece isn't the lightest, but it is light, compresses well and is very cheap.