Go! My DD was 18 months when she went, and her older brother was 5. We took it easy the day of the party and made sure she got a nice, long nap in the afternoon.....we did not even go to a park that day. She had a great time. She eventually fell asleep in her stroller after the fireworks, which was no big deal.
We did not stay out until midnight of course, but we did stay out later than we normally do (our kids also usually go to bed at 8:00 or 9:00, even when we are at Disney). We just slept in the next morning. Since we made sure they were well rested the day of the party, and we slept in the next day, they did GREAT! We would not have been able to do that every night of our trip or they would have been sooooo cranky, but they handled one night just fine.
We did the same for the P&P party when she was 2 1/2, and it worked out great again.
If you overdo it the day of the party and the child starts off the evening already tired, and you have an early morning planned for the next day, then you will run into problems.