Breakfast isn't bad at all. Especially when you're the first family in the door at 7AM and no one shows up for the next 30 minutes. Years ago our youngest got picked as chef-of-the-day that morning. I think the whole staff was with us in the pic along with all the characters. So breakfast has a soft spot in all our hearts.
BUT. It's just breakfast. The menu selection, I don't think, varies much nor is the selection as large as dinner. Now that the kids are older, they crave the evening meal. Myself, it's the mash potatoes - none better on property. My oldest can't get enough of the shrimp or prime rib. I think the wider variety later in the day makes a difference in the whole experience.
What I can't get over are the crowds. Make sure, if you don't do breakfast, that you have your ADR's sorted out. In the last 3 years or so, it's started to get really ugly outside with people waiting, stepping all over each other and nastiness at the outside check-in line, especially the walk-ups. That's the only thing that spoils the experience.