Temps can swing a lot that time of year in the Ozarks. The Ouachitas will generally be a bit warmer but you can expect it to get down in the mid-low 40ºs and give or take 10º depending on which way the wind is blowing.Here's the Kiamichi River Conditions page that shows the current water levels.
You might find yearly averages if you poke around that site a bit.
Should be a great trip. That's the perfect time of year to be there!