I don't spend much time at the resort either (I never use the pools or anything like that), but the resort theming is still a big part of the experience to me.
My issue with the Swan and Dolphin is the price, though. They're not that much cheaper than the Disney deluxe resorts, and cost more than the moderates. If I'm going to pay those prices, I want the theming. Otherwise I could just stay at one of the Disney Springs area hotels for much less.
They only make sense to me if you have a bunch of Marriott points and can stay there for a reduced rate or free.
I haven't looked at the prices recently, but when I have stayed there the prices were dramatically less then the other Boardwalk area resorts, so I was basically getting Deluxe amenities for the price of a Moderate.