Did Walt tell you that? He wore a coat and tie regularly on weekdays until his death. And when it was time for public appearances on TV or in his park, he was almost ALWAYS in a necktie.
The only photo I know of with Walt in Disneyland without a necktie on was this circa 1957 photo shoot for Life Magazine. Walt and his wife and their lovely daughter were photographed riding a few rides, and Walt is wearing a casual shirt and a sport coat. That's about as casual as Walt ever got in public.
This was Walt's usual dress code on working visits to Disneyland, seen here on Main Street USA and mobbed for autographs by Disneyland guests circa 1962. Walt had been driving Disneyland's Fire Engine down Main Street, but had to pull over to the curb to sign autographs.
Can you imagine Bob Iger doing this?!?
We also just used this photo over in the Disneyland forum here, but it works here as well. It's Walt Disney in 1966 visiting WDI's Glendale campus about 15 minutes from the Burbank Studios. Judging by the angle of the sun and Walt's lightweight (seersucker perhaps?) jacket, it's Summer '66. He's there to test out a mock-up of the PeopleMover that's going to be installed at Disneyland in 1967. He's wearing a coat and tie as he rides around on the test train at Imagineering. This wouldn't have been a photo shoot designed for public consumption, due to the ugliness of the setting and un-magical aspect of the testing process. It's just what Walt chose to wear to work that day.