Are you paying all of the expenses for the group? If you've invited everyone on the trip as a gift, I think you probably have to include them in everything unless you originally specified that you'd want some time separate from them. I know that when I treat family or friends to a trip it's because they simply can't afford it. If they are truly broke and have no spending money I feel like we do have to do everything together especially when it comes to meals because otherwise they probably can't eat.
If this is not the case and each adult is paying their own way and for their own children, here is what I do. I make an itinerary of everything that I want to do each day- which park I'll go to, where I will eat all meals, what extras I will be adding on (excursions, tours, boat or equipment rentals, special fireworks viewing, dessert parties, holiday parties and events). I print out a list for each day giving details including the approximate cost as well as dress code if applicable. I also provide a deadline for when I want to make the reservation for each meal/activity. Everyone has to chime in by each deadline to be included. This way I'm upfront about my plans and expectations and each family member or friend can choose to be included knowing exactly what the costs and expectations are for their participation. I never feel guilty about it and I never hide my plans because everyone understood well in advance of the trip. I find that it's easier to avoid hurt feelings or resentment if I communicate clearly and openly in advance.