Personally, some of the best food I've had has been at WDW. Not only is their food good, but in the "real world" most places do not go as above and beyond for people with allergies/restrictions as WDW does. (I don't have food allergies but I do have two digestive disorders, however I am a glutton for punishment so I just carry tums around LOL.)
We only had two bad experiences last time we went, both at EPCOT. We did a QS for lunch in the Land pavilion....some Asian food court style that was awful, and then Coral Reef for dinner. HOWEVER, we sat next to the tank and had the best waiter I've ever had (plus free dessert since it was our honeymoon) so we didn't mind too much. Other than that, everything we ate was great, and even Cosmic Ray's was good, although it was packed so the "condiment bar" was empty. Now we did eat at places like Narcoossee's, California Grill, etc, but we did a lot of our resorts food court and a good mix of TS and QS, since we did the dining plan. I would say the ONLY place we wouldn't go again is the food court in the Land pavilion...my husband doesn't like the sauce at Tony's but he's Italian and his mom is a great cook so he's biased. But I genuinely love WDW food (I mean come on, the turkey and stuffing at Liberty Tree??? SO GOOD.)
Of course when you do a lot of QS, burger and fry style joints, you might be disappointed. Especially depending on time of day/how busy it is, but isn't that kind of true for places like that anywhere?
The only time we ate off property was the first NFL game of the season, so we could watch the Patriots play, at Buffalo Wild Wings, with friends.