We have stayed at both WL and the Polynesian.
Based on the number in your party, the ages of your children and how you plan to go to the parks in the morning, take a mid-day break, and then back to the parks later, I whole heartedly recommend the Polynesian to you over the Wilderness Lodge...