Honestly, I think you'll be fine. We did a multi-generational, 6-person trip in May. The key was to know ahead of time who-could-do-what, keeping height restrictions and personal preferences in mind, and develop a touring plan based on that. With a plan pre-set, following it was a breeze (we departed from it when circumstances dictated, but it was nice not to have the bulk of the planning done ahead, so I didn't have to engage in any high-level thought while on vacation!) and we spent 90% of the time together.
As an example: when we toured Adventureland, Liberty Square and Frontierland in the Magic Kingdom (the afternoon and evening of our arrival day), the plan looked something like this:
1 - Swiss Family Treehouse
2 - Jungle Cruise w/FP+
3 - Enchanted Tiki Room
4 - Pirates of the Caribbean
5 - "Thrill ride/tall enough" crowd (3 adults and 1 child) on Splash Mountain w/ FP+ -- "Non-thrill-riders" (1 adult and 1 child) visiting the nearby Aladdin's Magic Carpets w/FP+, Aladdin meet-and-greet and/or Adventureland "7 Seas" Scavenger Hunt
6 - "Thrill ride/tall enough" crowd on Big Thunder w/ FP+ -- "Non-thrill-riders" grabbing Dole Whips for the group and reserving a 4th FP+ for Haunted Mansion
7 - Rendevous in Liberty Square to enjoy our Dole Whips together
8 - Country Bear Jamboree
9 - Hall of Presidents
10 - Dinner at Liberty Tree Tavern
11 - Haunted Mansion
12 Begin non-headliners in Fantasyland - its a small world, Mickey's Philharmagic, etc. (reserving Tom Sawyer Island and/or Liberty Belle for another day)...
As you can see, this touring plan (which we covered between 2pm and 8:30pm) had us separated for only 30 minutes! The vast majority of attractions in every park are suitable for all ages, and for those that aren't, there's almost always a small-child or motion-sickness-proof option nearby. I don't think you'll have much trouble keeping everybody happy. (The biggest advice I can offer in that regard is to make sure that EVERYBODY gets a nap every afternoon, so no one is melting down by the end of the day, and everyone is "up and at 'em" for rope drop arrivals in the morning. All you'll "miss" between 1pm and 4pm is the worst heat and crowds of the day!)
FWIW, here are what we sometimes use as the "split up" alternatives in each park (with the child-friendly alternative in bold):
Magic Kingdom: Space Mountain/PeopleMover, Barnstormer/Casey's Splash Zone or Pete's Silly Sideshow, Splash/Aladdin M&G, Enchanted Tiki Room, Aladdin's Magic Carpets, Adventureland Scavenger Hunt or Splash Moutain play area, BTMRR/Liberty Belle, Tom Sawyer Island or Country Bear Jamboree
Epcot: Mission:Space/Mission:Space play area (between the attraction exit and gift shop -- get in the queue with the rest of the family and take the "chicken exit" just before boarding the ride vehicle), Test Track/Innoventions West or Character Spot, Soarin'/Living with the Land or Circle of Life film
HS: Sunset Blvd. attractions (Tower of Terror, Rock'n'Rollercoaster) / Animation Courtyard attractions (Character M&Gs, Disney Jr. show, Little Mermaid show, TSMM if arriving at rope drop), Star Tours/Jedi Training Academy Show or whatever film preview is showing in the auditorium nearby
AK: Dinosaur/Triceratop Spin or The Boneyard play area, Expedition Everest or Kali River Rapids/Maharajah Jungle Trek or Rafiki's Planet Watch