Turf Club at Saratoga is a very nice restaurant, and hardly ever packed. I'm not necessarily saying you should stay at SSR (though I like it a lot), but since it's just for a weekend, and ressies are slight, you might be better off staying at SSR, so you can just walk from your room to the main house for dinner. Personally, were I flying solo, I'd probably do all (or almost-all) Quick Service and save money for the next time.
But bottom line is, it's your vacation and yours alone. You have no one else to please, and you know you better than we do, so you should choose the resort you enjoy the most. The other external factor I'd consider is where you intend to spend most of your time. You know the usual advice - if you're a MK kind of guy, try for a monorail resort or WL, if you prefer Epcot and/or the Studios go for one of the resorts in-between the two, AKL if you prefer AK, SSR or OKW if you want to do a lot of time downtown. And if you think most of your time will be spent lounging at your resort, then as I suggested before, go for the resort that you hold the fondest.