I think you might be overestimating the percentage of the general public that cares for WDW. Obviously, I don't have real numbers, but I'd be deeply surprised if even half of your average adults were interested in WDW. I think if you polled the general public, the overall sentiment would be something along the lines of 'It's expensive and not really something you would do without children.' If you look at conference goers who don't have their children with them, the numbers are probably even lower. Most of these people are either all-business or they want to enjoy the type of activities that aren't practical with their families in tow.
I've gone to a number of conferences at WDW, including one I attend annually that alternates between WDW and Vegas. The general sentiment when they're held at WDW is indifference, followed by disappointment, bewilderment, or annoyance. The folks excited to be at Disney are a definite minority. Most of the things we enthusiasts take for granted are more than a little annoying to conference attendees. The drinks are expensive and weak, the food is expensive and bland, there are children everywhere, room service isn't 24/7, etc.
During the last conference I attended at WDW they rented out Hollywood studios for an evening as the big tentpole entertainment of the conference. The whole park, just for the 1500-2000 conference attendees. They put open bars all over the park. A handful of restaurants were open and the food was all complimentary. All the rides were open. About half of the conference bothered to show up. As many as not preferred to just go offsite and eat at a 'normal' restaurant, go to a bar, have some drinks, etc.