I'm surprised at the few reports of trouble with the rider switch process. I've had no trouble at all. Each ride does it differently, so ask the CMs. If there is fastpass, they usually just give you a fastpass, so one rides throught the normal line and the other then goes throught the FP line. If there is no fastpass at the ride, there will be another way to do it. If you get a dumb CM who doesn't know the process, just find another!
On most rides, this works fine. Someone mentioned Soarin as a problem, and they are right. Because of the loading process for Soarin', even a trip through the FP line takes about 45 minutes start to finish, so doing it twice...twice as long. But everything else works great.
By the way, the 4 1/2 year old will likely be tall enough for most rides. My 4 year old rode Soarin', Test Track, Thunder and Splash Mountains, Kali River Rapids, etc.