It depends on how important the characters are to you. This is the only place that has some of those characters (although I saw Stich at the MK on the last trip, I've never seen Lilo anywhere else), so if you and your party absolutely have to see those characters, then there you go - breakfast is your only choice. But the food is decidedly ordinary breakfast food.
Having said that, dinner is great. The kids get to play games (like a hula-hoop contest and other fun stuff), and it's a great show by the kids for the grown-ups as well. The food is pretty good - not the best I've had - but certainly plenty of it. Eat all you care to, and they do not make you wait for it. (although the service style is odd - it's like Brazilian barbeque - I've never heard of this style of service being associated with polynesia - Oh, well; it doesn't really matter).
Anyway, why not do both?