Early to mid-November is a great time to go. Peak hurricane season is over, crowds are usually (but not always) a little down as not as many families travel due to the holiday season (breaks) being right around the corner, and the temperatures are much more reasonable. In mid-November, you get to see the parks decked out for the holidays. It is a little odd experiencing the Christmas season to such a strong degree that early in the year but still neat nonetheless.
I have heard there is a small uptick in crowds on Veteran's Day weekend. And like any time of the year, any park that has EMH that day will be busier than usual, especially if it is a Saturday or a Sunday.
The wife and I were there November 14 - 21 last year. The first 4 days were absolutely perfect in terms of weather. I don't think it rained until our 5th day, which was our SeaWorld day. It was one of those typical all afternoon Florida downpours. Day 6 saw some late afternoon/early evening rain but it let it in time for viewing of IllumiNations. Day 7 at Animal Kingdom was perfect. Day 8 at Magic Kingdom was probably the worst and even then it really didn't start opening up until right before MSEP kicked off. Some poor college marching band was and just made there way around the hub when it just started coming down Old Testament style. The parade was cancelled and the wife and I eventually made our way over to CoP to escape the deluge. Fortunately, the later performance of Wishes! was able to go on as scheduled.