We took our oldest daughter when she was weeks away from her 3rd birthday, and it was great timing for us. In fact, we're doing the same thing with her little sister in Feburary! For us, it was great that she was down to one consistent nap, so we just took an afternoon break everyday. And she *loved* it (once she figured out that she did indeed have to take her nap!).
We were down in the Orlando area for several months when DD 2 was an infant and the older one was 3 1/2. We went to WDW for a few day trips.....honestly it was a bit miserable for the baby. I felt really bad because she was quite heat intolerant. Thank goodness for air conditioned Baby Centers! That's where she and I spent most of one day! I had the opportunity to take DD 1 for a day and leave DH and baby at home...that day was much better for all of us.....
Having said that, only you know your family and baby. I do know how very hard it is to wait for a WDW trip! For us, knowing our daughter, we decided to wait and go for that 3rd birthday again.