My husband doesn't fit into a stroller well, at all,
and my tall 7 and 8-year-olds would look pretty ridiculous in them with their limbs dangling out in all directions (indeed, I haven't tried to put my husband or children in a stroller for any reason in the past four years), so we always return to our hotel for at least 2-3 hours for a nice nap in the air-conditioning and sometimes a refreshing dip in the pool. We do stay on-site, so getting there is a relatively painless, and often pleasantly scenic, trip.
I am of the opinion that mid-day breaks and naps aren't just for kids, either! We never take naps at home, but at WDW, we're up early and hitting the parks hard in the morning (on our legs, sans stroller, even when the children were as young as four), so all four of us are more than ready for a rest by lunchtime, which is when the crowds, lines, noise and heat are reaching their peak. I have a slight hearing impairment that makes it difficult and stressful for me to function in big, noisy crowds for very long, so leaving at that point is especially nice for me, and by that hour, the kids are pretty wound up and overstimulated, as well.
Once we've taken our break, we can return to the parks, refreshed, recharged, relaxed and ready for a few more hours of fun before bedtime, just as the temperature is starting to cool, wait times are beginning to diminish, and the overtired crowds are beginning to drift toward the exit. I've never felt that the time we spent on our mid-day break was "wasted" in any way. Between rope drop arrivals and intelligent use of FP+, we've never missed out on anything we wanted to do, even with "only" two 4-5 hour touring sessions per day, and (knock on wood), none of us has ever had a meltdown over anything! I'm pro-break, all the way!