It also depends on how many trips you plan to make.
Obviously, if you are DVC members and assume that every vacation will be at WDW, then bring them any time.
If WDW will be once or twice in a lifetime, then I would wait until they are out of diapers and strollers (although THAT age is hotly debated!).
But by far, the most important criterion is when do YOU want to take a child.
Children, especially very young ones, severely limit what you can do. You will skip lots of rides, and if you are in tune with your child, you will spend lots of time in the resort for naps and swimming.
As each grandchild turned 5, they got a trip to WDW from Granddad. It was a great age--no diapers, no strollers, no fear of dark rides, no fear of self-fllushing toilets (I'm not kidding!) no fear of characters, a willingness to play along, a desire to believe, stamina for 6-7 good hours of walking, and the verbal skills to express their joy and wonder.
You are the parent. You get to make these decisions. And it comes down to what you feel you can deal with.