I've been going the first week of May for the last nine years (it's our wedding anniversary) and have found it to be just right, weather- and crowds-wise.
As everyone else has said, it's very warm but not rainy and not tooo crowded, either. I've also made trips in February, March, April, June, September, and November.
Feb/March was great weather, but startling to come back to NY in the winter after having spent only a few days in Florida. My one April trip was at the end of April, so it might as well have been May! I found it rainy in September (only one trip), November is usually nice--chilly at night--but I made up for that at the Epcot Food & Wine festval! I would never go back in June again, rainy and crowded. Having said that, I find myself amazed to be preparing to pack my suitcase for a trip to WDW in eight days. Could there be a worse time to go than the week leading into Labor Day? I'll find out, that's for sure! It's a good cause, though--I'm taking my Mom for her 70th birthday!
There you go--waaay more information than you asked for!