Others have already said how I feel, but as long as you realize you're going for your benefit (not for the child who's too young to remember the trip), and as long as you realize that you'll nonetheless have to tailor most, if not all, of the trip around your child's wants and needs and schedules...go whenever you want to go. And have a great time.
I've always been amused by people who take their very very young children to WDW and insist they're doing it "for the kids." If that were the case, save the money, do something local and go to WDW when he or she would be old enough to not only appreciate it but be more involved with the actual vacation. But if YOU want to go, and you happen to have a child, have at it.
We went with our baby, who turned only 5 months old during the trip. There was a lot about it that was difficult, but we always wanted to go in December, my wife (who is a school teacher) is not the type to take days off beyond sick days where she or the baby is really sick, and the idea of holiday crowds makes each of us groan a little. She took some unpaid family leave, so we figured, this may be the only time between now and retirement we can go in early-mid December before crowds were crazy. And despite the many difficulties we had some incredible times, too, the never-gonna-forget-this kind. We had fun, it was just work. Be aware of that, and you should do fine.