Cheap and ultralight gear list Mark Verber's Cheap Gear Page
The outlet websites for backcountry.com and REI.com, Sierra Trading Post, your local thrifts stores, steepandcheap.com, and borrowing from other people.
Using a tarp instead of a tent. Beginners can encounter bad weather from time to time, and tarps take a bit of practice, but generally when you start out, you watch the weather and only go one or two nights (if you're starting smart).
Using what you have. Headlamps are cool, but seems pretty much everyone has one of those little cheapo flashlights. They work.
Recycling. Use a plastic bowl from a Country Time Lemonade container. Use a plain ol' fork and spoon from the thrift store, or that old set of silverware, or snitch one from McDonalds. Make a stove out of pop cans or use a Fancy Feast can. Make a windscreen out of a dollar store aluminum pan.
Trash bags can do anything. I have used them for ground sheets (seven bucks gets you a roll of contractor bags big enough to use as solo tent groundsheets or just to throw down under your pad). Also pack liners, or something to bundle up sticks for firewood to carry back to camp, or to stash wet boots in when you think it's going to be in the teens and want to put the boots in your footbox so they are not ice blocks in the morning. Contractor bags go in my daypack to be an emergency bivy. Poke holes and you have an emergency rain poncho.