I think they simply believe that stereotypical Disney adults won't take their job seriously and be distracted at work. Or worse, violate company policies about spreading rumors, trying to pawn goods on ebay, or not treating their job like a job. Or maybe there's a established history of super fans being thrilled at first to get the job, only to burn out quickly when they realize working at WDW is far less magical than vacationing there.
In theory, the ideal candidate is someone who has some fundamental knowledge and appreciation of Disney, but has the responsibility and work ethic to know when work is work and have the kind of customer service and task specific skills to deal with all types of guests and situations that the job requires.
Obviously, someone who has to be trained from the ground up on Disney lingo, products and standards is going to be a challenge, especially if they also don't take the job seriously, or see little value in it. This happens a lot with min wage front line employees and leads to high turnover, which is also a problem.