The chances of you seeing anyone from one of the parks ever again are slim to none. As others have said, it's YOUR vacation, do what you want wear the ears and ENJOY.
I know it's hard to deal with people staring, I deal with it daily because of my Special Needs Daughter. Some day's I get a bit...