There are 2 main reasons, which are interrelated, yet distinct.
Spaceship Earth is an absolutely gorgeous design. The simplicity of the monolithic geodesic sphere is breathtaking. It needs nothing to add to it, as its essentially perfect the way it is. Many of us feel that adding the wand robs it of its elegant beauty and power. It makes it look cheap, as the hand/wand is essentially a tacky 2-dimensional picture bolted a very industrial looking metal scaffold. The curvy font used for the word Epcot is non-standard and clashes with that aforementioned simplicity, not to mention that its pointless to have a huge sign, guest should know what park they're in.
More importantly, Mickey's hand and wand come from the Sorcerer's Apprentice, which is about music and magic. Epcot is the "real-world" park of technology, and international cultures and the future of man-kind. Thematically, its completely separate from everything else in the park; there's absolutely no connection between it and anything else found within. The globe of Spaceship Earth represents the Earth itself; the communication theme of the show inside connects it to all other Futureworld pavilions through its discussion of technology, and to The World Showcase through the ideals of world-peace and breaking down communications barriers. Adding a Mickey arm cheapens it, it degrades that message. Yes, Mickey Mouse is a powerful international symbol beloved the world over, but this theme park isn't about him, its about us, all of us.