Here is a basic list of clothes you already own or can pick up cheaply for summer (read 40F and above) backpacking.

A swim suit with mesh liner as shorts (no underwear necessary)
Nylon warm up pants
A synthetic athletic shirt (e.g. Target's DuoDry)
a polyester fleece jacket or pull over(read the label some fleece now contains cotton)
a pair of running shoes
two pairs of socks (wool are the best but people use Thorlos, Cool-Max socks from Target/Wall-Mart and even nylon dress socks)
a knit hat for warmth
a baseball cap for sun
a rainjacket or poncho (You can pick up a cheap poncho just about anywhere but you might want to take a wind jacket too if you go this route)

Check your draws and closet you probably have everything here except the rainjacket and the wool socks.