If your bag has a "lower limit" rating of 19*F (that seems awfully low if the "comfort" rating is 32*; the difference between the two is usually about 7-9*F), it was tested using a 1" thick foam pad with an R-value of about 5.0. Also the testing dummy wore a base layer and a knit cap. Some of these EN13537 ratings seem a bit iffy, although they are far superior to the ratings dreamed up by manufacturers' marketing departments.

From my own experience, I'd agree that your problem is the pad. Either you spend the money to get a pad with R value of 5 or above, or you keep adding pads (plus weight and bulk!) to get the total R value up to 5. Do some research and then consider waiting for REI's next 20% off coupon in March, which will help. In the meantime, also watch for sales at other outlets.

Because the air in an inflatable pad takes on the temp of the surrounding air, you want your additional insulation on top of, rather than under, the NeoAir.

I had the same issues some years back with a standard NeoAir (with which I never could get comfortable) and one of those 1/8" thick foam pads from Gossamer Gear. It got down to 18* two nights in a row on a late September trip. I was in my Western Mountaineering Ultralite (comfort rating about 24*F, lower limit about 17*F). I was wearing all my insulating clothing over a vapor barrier suit (my non-breathable rain gear). I also had a nice warm dog curled up against my feet. I was actually sweating on top, but shivering underneath! It was a tiny bit warmer with the foam pad on top of the NeoAir rather than underneath, but with only 1/8" of foam, not enough to do the job.

Another interesting item I found out back then is that if, like me, you want your inflatable pad nice and squishy, underinflating it reduces the R value by about a point.